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EDUCATION RESEARCH: IDENTIFYING 
EFFECTIVE PROGRAMS TO SUPPORT 
STUDENTS AND TEACHERS 


Wednesday, November 16, 2011 
U.S. House of Representatives 
Subcommittee on Early Childhood, 
Elementary and Secondary Education 
Committee on Education and the Workforce 
Washington, DC 


The subcommittee met, pursuant to call, at 10:01 a.m., in room 
2175, Rayburn House Office Building, Hon. Duncan Hunter [chair- 
man of the subcommittee] presiding. 

Present: Representatives Hunter, Petri, Platts, Foxx, Hanna, 
Barletta, Roby, Kelly, Payne, Scott, McCarthy, Holt, Davis, and 
Woolsey. 

Staff present: Jennifer Allen, Press Secretary; Katherine Bath- 
gate, Press Assistant/New Media Coordinator; Heather Couri, Dep- 
uty Director of Education and Human Services Policy; Lindsay 
Fryer, Professional Staff Member; Krisann Pearce, General Coun- 
sel; Mandy Schaumburg, Education and Human Services Oversight 
Counsel; Dan Shorts, Legislative Assistant; Linda Stevens, Chief 
Clerk/Assistant to the General Counsel; Alissa Strawcutter, Deputy 
Clerk; Brad Thomas, Senior Education Policy Advisor; Kate 
Ahlgren, Investigative Counsel; Daniel Brown, Junior Legislative 
Assistant; John D’Elia, Staff Assistant; Jamie Fasteau, Deputy Di- 
rector of Education Policy; Ruth Friedman, Director of Education 
Policy; Brian Levin, New Media Press Assistant; Kara Marchione, 
Senior Education Policy Advisor; Melissa Salmanowitz, Commu- 
nications Director for Education; Laura Schifter, Senior Education 
and Disability Advisor; and Michael Zola, Senior Counsel. 

Chairman Hunter. Good morning. A quorum being present, the 
subcommittee will come to order. 

Welcome to today’s subcommittee hearing. I would like to thank 
our witnesses for joining us today. We look forward to hearing your 
testimony. 

Providing more information about educational quality to families 
and communities is essential to improving K-12 schools in America. 
We are here today to discuss the value of education research, ex- 
plore the appropriate level of federal involvement, and examine 
ways to improve current law to provide more immediate and rel- 
evant data to parents and educators. 

(l) 
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Since the enactment of the Education Sciences Reform Act the 
federal government has played an important role in supporting re- 
search and program evaluations and gathering data about edu- 
cational practice and the nation’s schools. Today, federal expert 
panels and research centers offer support to state and local organi- 
zations that perform education research. 

The responsibility for education research is shared by both fed- 
eral and nonfederal organizations in an effort to examine the qual- 
ity of existing programs, develop and test innovative practices, and 
ensure the effective use of taxpayer dollars. 

The resultant data allows teachers, parents, and officials to gain 
a greater understanding of successful interventions, school per- 
formance, and student achievement. For example, the Institute of 
Education Sciences established the What Works Clearinghouse to 
provide educators, policymakers, and the public with a central and 
trusted source of scientific evidence of what works in education. 

Information from the clearinghouse showed the “I CAN Learn” 
curriculum resulted in significant achievement gains for 8th grade 
and math students. However, the What Works Clearinghouse 
needs improvement, especially in providing clear direction on ap- 
plying research to classroom practices. 

Education research has also helped us identify programs that are 
not helping students succeed. Particularly in these times of trillion 
dollar deficits and record debt, congressional leaders must be care- 
ful stewards of taxpayer dollars. 

We can all agree on the need to dedicate federal education funds 
to the most effective programs; if research and data show a pro- 
gram is not working we should get rid of it. That is why my col- 
leagues and I introduced legislation to eliminate more than 40 inef- 
fective or duplicative programs as part of our K-12 education re- 
form package. 

Through the Education Sciences Reform Act and related initia- 
tives we have made great strides in assessing the quality of K-12 
schools, protecting taxpayers’ investment, and identifying success- 
ful education practices. However, as we look toward reauthoriza- 
tion of this law we must acknowledge the challenges facing edu- 
cation research and the Institute of Education Sciences. 

For instance, we must find better ways to help states and school 
districts translate the best research principles into classroom prac- 
tices. Existing research centers designed to provide technical assist- 
ance to states and districts need to do a better job sharing informa- 
tion to help local education officials identify and implement the 
practices and programs that are most likely to work for their stu- 
dents. 

Another challenge exists in establishing a more collaborative re- 
lationship between the director of the Institution of Education 
Sciences and the secretary of education. Maintaining the autonomy 
and independence of the IES is extremely important; the director’s 
role must stay nonpolitical. However, more communication and 
data sharing between the two entities could ultimately lead to bet- 
ter, more effective federal education programs and initiatives. 

The witnesses here today have valuable insight into the ways we 
can ensure education research is beneficial to parents, teachers, 
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and students. I look forward to a productive and informative dis- 
cussion this morning. 

I will now recognize my distinguished colleague, Rush Holt, for 
his opening remarks. 

[The statement of Mr. Hunter follows:] 

Prepared Statement of Hon. Duncan Hunter, Chairman, Subcommittee on 
Early Childhood, Elementary, and Secondary Education 

Providing more information about educational quality to families and commu- 
nities is essential to improving K-12 schools in America. We are here today to dis- 
cuss the value of education research, explore the appropriate level of federal involve- 
ment, and examine ways to improve current law to provide more immediate and rel- 
evant data to parents and educators. 

Since the enactment of the Education Sciences Reform Act, the federal govern- 
ment has played an important role in supporting research and program evaluations, 
and gathering data about educational practice and the nation’s schools. Today, fed- 
eral expert panels and research centers offer support to state and local organiza- 
tions that perform education research. The responsibility for education research is 
shared by both federal and non-federal organizations in an effort to examine the 
quality of existing programs, develop and test innovative practices, and ensure the 
effective use of taxpayer dollars. 

The resultant data allows teachers, parents, and officials to gain a greater under- 
standing of successful interventions, school performance, and student achievement. 
For example, the Institute of Education Sciences established the What Works Clear- 
inghouse to provide educators, policymakers, and the public with a central and 
trusted source of scientific evidence of what works in education. Information from 
the Clearinghouse showed the “I CAN Learn” curriculum resulted in significant 
achievement gains for 8th grade math students. However, the What Works Clear- 
inghouse needs improvement, especially in providing clear direction on applying re- 
search to classroom practices. 

Education research has also helped us identify programs that are not helping stu- 
dents succeed. Particularly in these times of trillion-dollar deficits and record debt, 
Congressional leaders must be careful stewards of taxpayer dollars. We can ali 
agree on the need to dedicate federal education funds to the most effective pro- 
grams; if research and data show a program is not working, we should get rid of 
it. That’s why my colleagues and I introduced legislation to eliminate more than 40 
ineffective or duplicative programs as part of our K-12 education reform package. 

Through the Education Sciences Reform Act and related initiatives, we have made 
great strides in assessing the quality of K-12 schools, protecting taxpayers’ invest- 
ments, and identifying successful educational practices. However, as we look toward 
reauthorization of this law, we must acknowledge the challenges facing education 
research and the Institute of Education Sciences. 

For instance, we must find better ways to help states and school districts trans- 
late the best research principles into classroom practices. Existing research centers 
designed to provide technical assistance to states and districts need to do a better 
job sharing information to help local education officials identify and implement the 
practices and programs that are most likely to work for their students. 

Another challenge exists in establishing a more collaborative relationship between 
the Director of the Institute of Education Sciences and the Secretary of Education. 
Maintaining the autonomy and independence of the IES is extremely important; the 
Director’s role must stay non-political. However, more communication and data 
sharing between the two entities could ultimately lead to better, more effective fed- 
eral education programs and initiatives. 

The witnesses here today have valuable insight into the ways we can ensure edu- 
cation research is beneficial to parents, teachers, and students. I look forward to a 
productive and informative discussion this morning. 


Mr. Holt. Thank you, Mr. Chairman, and thank you for the 
hearing. 

And I am pleased to welcome the witnesses here today. I think 
we will learn a lot. I thank you for taking time to provide us with 
guidance on how to use data and research to improve educational 
practices at large and individual student performance. 
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A few years back I visited an elementary and middle school in 
Union City, New Jersey to learn more about innovations in a school 
district that had been troubled, and how they used data to improve 
student achievement. Union City is what we call in New Jersey an 
Abbott school district. It is a dense urban district with an over- 
whelming majority of English language learning students, and yet, 
to the surprise of many education experts, the district is meeting 
or exceeding state standards now. 

They did a number of things to accomplish this, but one thing 
in particular they did was to provide frequent evaluation of all stu- 
dents and shared the test data immediately with teachers. Union 
City teachers were able, then, to tailor their instruction to meet 
each student’s individual needs. Data showed that teachers and ad- 
ministrators could identify trends that could be addressed system- 
ically and individually, and this approach of continually using data 
to inform instruction helped the students do far better than pre- 
vious classes of students had done. 

Now, each of us thinks we are an expert on education because 
we were students. We have to guard against that, and we have to 
remember that there are things that we can learn about how peo- 
ple learn. And we need data, we need evidence, we need research 
to help us understand how people learn and how we can improve 
instruction. 

The Educational Sciences Reform Act was intended to provide for 
the improvement of federal education research, statistics, evalua- 
tion, and dissemination of data to inform education policy and edu- 
cation practice. It supports data-driven development and supports 
practitioners in understanding research and data from their 
schools. 

I really believe that it helps educators make decisions about their 
students’ learning experiences, and it helps states use research to 
identify successful instructional programs. It helps teachers and 
principals implement proven school improvement strategies, and it 
would help us if we would use those data and if we would use that 
research. The federal government plays an important role in sup- 
porting the research. Educators across the country need reliable re- 
search to enable them to make evidence-based decisions in the 
classroom, and they need data-driven systems that support instruc- 
tion. 

In reauthorizing the Elementary and Secondary Education Act, 
I hope we will maintain accountability for acceptable, adequate 
progress for all students. We can be more flexible with how stu- 
dents improve, how schools improve, and how we empower schools 
to use their data, if we make more use of evidence and drive evi- 
dence-based decisions. As the committee continues to work on the 
reauthorization of ESEA and ESRA I hope we will continue to pay 
attention to the role of research and data in improving student out- 
comes. 

I am going to reintroduce soon the Metrics Act to help improve 
data sharing and instruction at the local level. I think improved 
use of data can help all students do better, and I hope we will be 
able to include my legislation in any reauthorization of ESEA. 

Strongly held beliefs or ideological commitment should not trump 
data or evidence. If we want to make the best policy we need evi- 
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dence-based research. At the individual level, if we really want to 
hold schools accountable for adequate progress for each student 
they have to use data, and we have to see that it is used in the 
most illustrative way. 

So continued federal investment in educational research will be 
necessary if we are to ensure that all students receive a quality 
education that prepares them for life and further study. I hope the 
testimony today will provide us with some recommendations on 
how we can strengthen the ESRA and the federal investment in 
education research. 

Thank you, Mr. Chairman. 

Chairman Hunter. Thank the gentleman. 

Pursuant to Committee Rule 7c, all subcommittee members will 
be permitted to submit written statements to be included in the 
permanent hearing record, and without objection, the hearing 
record will remain open for 14 days to allow statements, questions 
for the record, and other extraneous material referenced during the 
hearing to be submitted in the official hearing record. 

It is now my distinguished pleasure to introduce our panel of wit- 
nesses. Dr. Grover J. “Russ” Whitehurst is the director of the 
Brown Center on Education Policy at the Brookings Institution. 
Previously, he was the first director of the Institute of Education 
Sciences. 

Dr. Caroline Hoxby is the Scott and Donya Bommer Professor in 
Economics at Stanford University, the director of the Economics of 
Education program at the National Bureau of Economic Research, 
and a senior fellow of the Hoover Institution and the Stanford In- 
stitute for Economic Policy Research. 

Mr. Steve Fleischman is the deputy executive officer of Education 
Northwest, formerly known as Northwest Regional Educational 
Laboratory, the organization that has managed the REL Northwest 
Laboratory since 1966. He has also served as director of REL 
Northwest. 

Lastly, Dr. Eric Smith is the former commissioner of education 
for the state of Florida. Dr. Smith is currently a consultant to a 
number of state education chiefs and school districts on several 
education reform projects. 

Before I recognize each of you to provide your testimony let me 
briefly explain our lighting system. When you start it will be green, 
you will have 5 minutes; when you have 1 minute left it will turn 
yellow; and when it turns red we would ask you to wrap up as best 
as you can. After everyone has testified, the members will have 5 
minutes to ask a question of the panel. 

I would now like to recognize Dr. Whitehurst for 5 minutes. 

STATEMENT OF DR. GROVER J. “RUSS” WHITEHURST, SENIOR 

FELLOW AND DIRECTOR OF THE BROWN CENTER ON EDU- 
CATION POLICY, BROOKINGS INSTITUTION 

Mr. Whitehurst. Thank you, Mr. Chairman and members of the 
committee. I really appreciate the invitation to testify, and I am 
pleased that you have such a keen interest in education research 
and reauthorizing ESRA. 

Everyone in the room knows that education is important. It has 
been true in this country throughout its history. In fact, before we 
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were a country a first thing that a small, colonial village would do 
is set up a school once it had enough kids to require schooling. 

But in an age of globalization and the advent of a knowledge- 
based economy, the imperative for us to educate well is stronger 
than it has ever been. High quality education research is critical 
to the nation’s effort to deliver better education and a future of op- 
portunities to our citizens. Without good evidence we are destined 
to embrace education policies that move us forward, backwards, 
and sideways, and we are not even going to know in which of those 
directions we are heading. 

The Educational Sciences Reform Act, which originated in this 
subcommittee in 2001, made great strides towards improving qual- 
ity and independence of federally sponsored education research. 
Prior to that legislation the federal stewardship of education re- 
search was widely viewed as a failure. 

Since then, we have seen considerable progress in the quality 
and relevance of that research and evidence for that comes from a 
number of sources. Let me just give you a very short list of some 
things we know now that we did not know 10 years ago that are 
a product of the federal investment in education research. 

On teachers, we know that teachers vary dramatically in their ef- 
fectiveness. A very effective compared to a very ineffective teacher 
can create achievement gains for a child in 1 year that can wipe 
out a third of the achievement gap between white and black stu- 
dents, and you can see the effects of a very effective teacher in ele- 
mentary school all the way into adulthood, in terms of college-going 
and job earnings. 

On the organization of schools, we know now that no excuses 
charter schools in urban areas do a dramatically better job than 
traditional public schools in raising student achievement. 

On standards, we have learned that the quality of state stand- 
ards for what students should know, contrary to what seem to be 
reasonable assumptions, bear no relationship to student achieve- 
ment. The states with the best standards can have low levels of 
achievement relative to states with weak standards, and vice-versa. 

On the effectiveness of federally funded education programs, we 
now know that a significant number of those programs are not 
achieving their intended effects. 

And finally, on basic learning and instructional processes we 
have a whole list of things we have learned, including the inter- 
esting fact that testing students on the content of the classroom as- 
signments produces substantially more learning than the same 
amount of time spent restudying the material. 

So I could provide you a much longer list. There are many things 
we know now that we did not know 10 years ago. If knowledge is 
power we are in a much better shape than we used to be, and that 
augurs well for the future. 

ESRA is overdue for reauthorization. I will not take you through 
a to-do list for reauthorizing the law. Let me simply say it is a 
pretty good piece of legislation; I think it needs some fine-tuning, 
and that is about it. 

Finally, I want to address the federal role in incorporating the 
findings from research into program mandates. No Child Left Be- 
hind uses the phrase “scientifically-based research” 111 times — I 
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counted — and it includes mandates for states and local education 
agencies to base their practices on research. The most extreme ex- 
ample is the now defunct program, Reading First, which dictated 
how early reading instruction was to be delivered at the classroom 
level at a very granular level. 

It is a fundamental mistake, in my view, for Congress to dictate 
how states and LEAs should use findings from research. Research 
is seldom definitive. Its reflection in statute and on-the-ground im- 
plementation is typically flawed, and our knowledge advances at 
too fast a rate for legislation to keep up. 

Instead of telling states and local education agencies what they 
should do and appealing to research as a justification, in my view, 
Congress should focus on creating incentives for practitioners and 
policymakers to incorporate research findings into their programs. 
Those incentives should be based around the performance of 
schools. 

When my grandfather learned about research findings that 
would help him generate a higher yield from his farm he didn’t 
need to be told by government that he had to utilize those findings; 
it was in his self-interest to do so and he did. Likewise, education 
providers will use research when it helps them do something for 
which they are accountable. 

There are two ways to fashion an accountability system that will 
create a demand for research findings. One is top-down regulatory 
accountability, as we have seen in No Child Left Behind. Wash- 
ington says, “Here are your targets for student achievement. If you 
don’t meet them the following things will happen.” 

The other approach is bottom-up marketplace accountability. 
Parents are given choices of where to send their kids to school. 
They get good information on school performance. Funding follows 
kids. Schools that aren’t performing well lose students and funding. 
The managers of those schools are motivated to improve their per- 
formance and seek solutions, including those from good research. 

I am in favor of a market-based approach to creating demand for 
research and I urge you to consider it in the context of the reau- 
thorization of the Elementary and Secondary Education Act. 

In conclusion, as a result of rigorous and relevant education re- 
search we know much more than we did about what works and 
what doesn’t in education than we did a few years ago. We have 
got a long way to go before we know enough to assure a good edu- 
cation to every student. 

We have started. We are making progress. I appreciate this com- 
mittee’s understanding of the importance of the work and the crit- 
ical role the federal government plays in advancing it. 

Thank you, Mr. Chairman. 

[The statement of Dr. Whitehurst follows:] 

Prepared Statement of Dr. Grover J. “Russ” Whitehurst, Senior Fellow and 

Director of the Brown Center on Education Policy, Brookings Institution 

Me. Chairman and Members of the Committee: I am Russ Whitehurst. I direct 
the Brown Center on Education Policy at the Brookings Institution. Prior to holding 
my present position, I was the founding director of the Institute of Education 
Sciences within the U.S. Department of Education. Before entering government 
service I had a long career as a researcher and academic administrator. 

Thank you for the invitation to testify. I am pleased that there is such interest 
and leadership in addressing the quality of education research in America. 
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Everyone in this room knows that education is important. I expect that all of us 
have had an experience with a teacher, a class, an educational institution, or 
through independent learning that has changed our lives. I certainly have. The 
American dream of opportunity and advancement and the educational system of the 
United State are inextricably connected. This has been true throughout our history. 
Indeed, well before the country was founded it was typical for colonial villages that 
had grown to more than a few hundred people to establish and fund a public school, 
with the first dating to 1639. Since that time, we have continued to value education 
and invest in it. But in an age of globalization and the advent of a knowledge based 
economy, the imperative to educate and educate well is stronger than it has ever 
been. The evidence that nations with a better educated populace experience higher 
growth rates is compelling, and during the current economic downturn the unem- 
ployment rate in the U.S. for young adults with just a high school diploma has been 
three times the rate for those with a college degree. 

High quality education research is critical to the nation’s effort to deliver better 
education and a future of opportunity to our citizens. Without good evidence on the 
condition of education, what works and what does not, fundamental processes of 
learning and instruction, and breakthrough instructional technologies we are des- 
tined to embrace education policies that move us forward, backward, and sideways 
without even knowing in which of those directions we’re heading. Without good edu- 
cation research, our approaches to education reform are more akin to fashion and 
fancy — the width of a man’s tie or the length of a woman’s skirt — than to anything 
that is rational and benefits from a systematic examination of evidence. 

Think of what federal investments in agricultural research have accomplished. My 
grandparents were farmers during the transition from the way things had always 
been to farming based on the knowledge produced by agricultural research. I re- 
member well my grandfather coming back from a meeting with an agricultural ex- 
tension agent excited about what new seeds and new approaches to crop rotation 
could do for the family farm. And because he was an early and eager adopter of 
research-based approaches to farming, he was always ahead of his neighbors in 
wringing a living from his land. These days America is the breadbasket for the 
world, largely because we invested in agricultural research and figured out how to 
disseminate the knowledge derived from that research to those who farm. We are 
on the cusp of a transformation of education to an evidence-based field that will 
have many similarities to the changes in agriculture that my grandparents experi- 
enced. The actions this Committee takes as it shapes the the federal role in edu- 
cation research will have far reaching effects on the quality and productivity of our 
schools, and through that on our economy and future. 

Mr. Chairman, the Education Sciences Reform Act, which originated in this sub- 
committee in 2001 and currently governs the education research enterprise at the 
Institute of Education Sciences within the U.S. Department of Education, made 
great strides towards improving the quality and independence of federally sponsored 
education research. Prior to that legislation, the federal stewardship of education re- 
search was widely viewed as a failure. To that point, in 1999 the National Acad- 
emies of Science came to the conclusion that: 

One striking fact is that the complex world of education — unlike defense, health 
care, or industrial production — does not rest on a strong research base. In no other 
field are personal experience and ideology so frequently relied on to make policy 
choices, and in no other field is the research base so inadequate and little used. 

Since the National Academies report and as a direct result of Education Sciences 
Reform Act we have seen considerable progress in the quality and relevance of edu- 
cation research. Evidence for this comes from numerous sources, not the least of 
which is the Office of Management and Budget. OMB’s most recent program assess- 
ment of the Institute of Education Sciences concluded that 

Since its creation by the Education Sciences Reform Act of 2002, IES has trans- 
formed the quality and rigor of education research within the Department of Edu- 
cation and increased the demand for scientifically based evidence of effectiveness in 
the education field as a whole. 

Let me give you some examples of things we’ve learned from recent education re- 
search that are very important to improving America’s schools and student achieve- 
ment. 

• On teachers 

Teachers vary dramatically in effectiveness — a very effective compared to a very 
ineffective teacher can create achievement gains for a child in one year that can 
wipe out a third of the achievement gap between white and black students. 

On-the-job performance is the single strong predictor of how good a teacher will 
be in the future — almost every other observable characteristic of teachers is at best 



9 


only weakly predictive of how they will perform in the classroom, e.g. whether they 
are regularly certified or not, were trained in a school of education or not, got a high 
or low score on a certification exam, received a lot of professional development or 
a little, and were mentored as novices or not tells us almost nothing about how ef- 
fective they will be as teachers. 

Most professional development programs for teachers are a waste of time and 
money. 

• On the organization of schools, choice, and competition 

No excuses charter schools in urban areas do a dramatically better job than tradi- 
tional public schools in raising student achievement. 

Armed with good information on school performance and the ability to choose 
schools, low-income parents choose better schools than the ones to which their 
school district would assign their children, and their children do better academically 
as a result. 

Schools that are subject to competition from other schools for students improve 
more than schools not subject to competition. 

• On standards, accountability, and curriculum 

The quality of state standards for what students should know bears no relation- 
ship to student achievement — states with the best standards can have low levels of 
achievement relative to states with weak standards and vice-versa. 

No Child Left Behind-type accountability for schools and districts raises student 
achievement modestly, with the effects focused in mathematics in the earlier grades. 

Curriculum choices can make a sizable difference — for example the difference be- 
tween using the most effective vs. the least effective elementary school mathematics 
curriculum, each costing about the same, is as much as a third of a year of learning 
over the course of one school year. 

Presently available educational technology programs as used in schools do not 
raise student achievement. 

• On the effectiveness of federally funded education programs 

There is a long-list of federal education programs that have no measurable effect 
on student outcomes, including: 

• The 21st Century Community Learning Centers Program (afterschool ) 

• Even Start 

• Head Start (for outcomes at the end of first grade) 

• Upward Bound 

• Reading First 

• On basic learning and instructional processes 

Spacing the occasions when students are asked to study related content rather 
than massing the study of that content into a short time frame remarkably in- 
creases learning and retention. 

Testing students on the content of their classroom assignments produces substan- 
tially more learning than the same amount of time spent restudying the material. 

I could provide many more pages of example of things we know now about edu- 
cation that we did not know 15 years ago. If knowledge is power, we’re in much 
better shape than we used to be and that augurs well for the future. 

The Education Sciences Reform Act is overdue for reauthorization. I will not take 
you through a to-do list for reauthorizing the law, one reason being that the Na- 
tional Board for Education Sciences has already generated such a list and I’m sup- 
portive of the Board’s recommendations. Let me simply suggest three principles that 
should underlie the reauthorization. 

1. If It Ain’t Broke Don’t Fix It — There are various groups, with the American 
Educational Research Association being the most prominent, that would have you 
make fundamental changes in the law that appeal to their interests. They would, 
for example, have you change the definitions of what constitutes rigorous research 
and evaluation to lower the methodological bar their members confront when trying 
to obtain federal grant money, and they would have you separate the National Cen- 
ter for Education Statistics from the Institute of Education Sciences in order to cre- 
ate another federal entity that they can try to influence and with which they could 
curry favor. The key question you should ask of advocates of any significant changes 
in the language in the bill is, “What evidence do you have you that the current lan- 
guage has had bad effects?” ESRA a pretty good piece of legislation and most efforts 
to change it are going to come from organizations that want a return to the wonder- 
ful days of yesteryear when education research produced little of value except fund- 
ing for education researchers. 

2. Independence Is Fundamental — One of the most important advances in the 
Education Sciences Reform Act was to create a greater degree of independence be- 
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tween the Department’s research arm and the political leadership of the Depart- 
ment. I led the Department’s research office for 8 years under two secretaries and 
multiple lesser political appointees. I had good relationships with the political lead- 
ership of the Department and we worked well together, hut I needed every bit of 
independence granted me by statute along with a fair amount of grit to keep my 
office and its functions from being politicized. I think this is in the nature of the 
beast rather than the personalities or political parties involved. Anything you can 
do to further arm future IES directors with independence from political direction 
will be positive. At the same time, the IES director needs to be inside the tent in 
order for the Department to benefit from education research and to have education 
research informed by insights on federal policies. 

3. The Regional Educational Lab Program (the RELs) Is Broken and Should be 
Fixed — The REL program goes back to 1966 and the very first Elementary and Sec- 
ondary Education Act. Since then, year in and year out, the RELs have pulled down 
a significant proportion of the total federal investment in education R&D with little 
to show of value from that investment and a lot to show that should be an embar- 
rassment. I don’t think any amount of tinkering with the legislative language that 
authorizes the RELs or aggressive intervention by the Institute of Education 
Sciences can fix what is wrong with the program. But there is a function the RELs 
are intended to serve that is desperately needed: helping states answer questions 
about the effectiveness and productivity of their own education programs using state 
administrative data. The goal of having statewide longitudinal education databases 
in every state was pursued vigorously in the George W. Bush administration. The 
Obama administration has added substantially to funding for this effort through the 
American Recovery and Reinvestment Act of 2009. In the near future all states will 
have data warehouses with longitudinal student achievement data linked to a vari- 
ety of education input variables. However, having data available and being able to 
use it are two different things. Only a few states have the staff capacity within their 
state education office to conduct analyses of longitudinal data to address policy 
questions. This means that most policy initiatives fly blind, both in original design 
and subsequent appraisal. RELs might be assigned through legislation to carry out 
this task, but they have multiple masters (including the federal government, their 
own boards, the governors and state legislatures in their region), they vary substan- 
tially in their capabilities, and they have no easy way to prioritize among various 
claims on their resources. It would be much better in my view to eliminate the REL 
program and substitute for it a research voucher program for state education de- 
partments. The current REL budget would be split among states, taking some ac- 
count of state population but making sure that smaller states receive a cut of the 
pie that is large enough to be useful. The states could spend their vouchers to con- 
tract for research on issues of high interest to them. The research plans and prod- 
ucts would undergo methodological review at IES to assure quality, but would other- 
wise be independent of the Department. The current RELs could compete for this 
work. If they could do the work well they would prosper. If they could not they 
would have to go into another line of work. It is a marketplace solution to a problem 
that has proven intractable to previous legislative and administrative solutions. 

4. You Get What You Pay For — Although federal budgetary support for education 
research has increased in the last decade, it remains a pittance when compared with 
levels of investment in research, evaluation, and statistics in other areas of the 
economy. For example, more than 40% of the discretionary budget of the U.S. De- 
partment of Health and Human Services is invested in knowledge production and 
dissemination through the National Institutes of Health, the Centers for Disease 
Control, the Food and Drug Administration, and many other operational compo- 
nents. In the U.S. Department of Education, the corresponding investment is less 
than 1%. In education research and development, no less than in R&D in health 
or transportation or communication or energy or agriculture, the public gets what 
it pays for. 

Finally, I want to address the federal role in incorporating the findings from edu- 
cational research into program mandates. NCLB uses the phrase “scientifically- 
based research” 111 times, and includes many mandates for states and local edu- 
cation agencies to base their practices on the findings from such research. The most 
extreme example is the now defunct program, Reading First, which dictated how 
early reading instruction was to be delivered at a very granular level based on re- 
search findings. There is no evidence that children are reading better as a result. 
It is a fundamental mistake, in my view, for Congress to dictate how states and 
LEAs should use findings from research. Even if the research were absolutely defin- 
itive, which it seldom is; and Congress could translate it into legislation without dis- 
tortion, which it can’t; and bureaucrats in the U.S. Department of Education could 
implement it unimpeachably, which is unlikely; science is dynamic. We shouldn’t ac- 
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cept a process that requires Congress to rewrite legislation in order to bring edu- 
cation practice in line with evolving research findings. 

Instead of telling states and local education agencies what they should do and ap- 
pealing to research as the justification, Congress should focus on creating incentives 
for practitioners and policy makers to want to incorporate findings from the best re- 
search into their programs. Those incentives should be around the performance of 
schools. If those who are responsible for the management of schools are held ac- 
countable for schools’ performance, and if research findings are both readily 
consumable and provide a obvious boost to school performance, then the research 
will be utilized. When my grandfather learned about research findings that would 
give him a leg up in the yield from his farm he didn’t need to be told by big govern- 
ment that he had to base his practices on that research. It was in his self-interest 
to do so because he was accountable for earning a living from his farm. Likewise, 
education providers will use research when it helps them do something for which 
they’re accountable. 

There are two ways to fashion an accountability system that will create a demand 
for research findings. One is top-down regulatory accountability as we’ve seen in 
NCLB — Washington says, “Here are your targets for student achievement. If you 
don’t meet them the following unpleasant things will happen.” The other approach 
is bottom-up market place accountability — Parents are given choices of where to 
send their children to school and good information on school performance. Funding 
follows the child. Schools that aren’t performing well lose students and funding. The 
managers of those schools are motivated to improve their performance and seek so- 
lutions, including those from good research. 

I’m in favor of the market-based approach to creating demand for education re- 
search and I urge you to consider it in the context of the reauthorization of ESEA. 

We know much more about what works and what doesn’t in education than we 
did 15 years ago as a result of advances in research, but our level of ignorance 
dwarfs our understanding by orders of magnitude. It has been so in the early years 
of the transformation of other fields to evidence-based practice. Moving education 
to a point at which our research base is sufficient to assure a good education for 
every student is the work of a generation, not of a few years. We’ve started and 
we’re moving in the right direction. I appreciate this Committee’s understanding of 
the importance of the work and the critical role the federal government plays in ad- 
vancing it. 


Chairman Hunter. Thank you, Doctor. 

Dr. Hoxby is recognized for 5 minutes. 

STATEMENT OF DR. CAROLINE HOXBY, SCOTT AND DONYA 

BOMMER PROFESSOR OF ECONOMICS, STANFORD UNIVER- 
SITY 

Ms. Hoxby. Mr. Chairman and members of the committee, thank 
you very much for inviting me to testify. It is an honor. 

The United States faces a bleak future if we do not improve the 
education of our population. The American industries that are still 
growing quickly and exporting are those that are most dependent 
on having educated workers, and if our economy is to grow fast 
enough to solve our fiscal problems we really need to have a more 
productive education sector — in other words, achieve more with the 
same amount of spending. 

As the Education Sciences Research — Reform Act greatly trans- 
formed education research and moved it much closer to the success- 
ful models that we associate with the National Science Foundation 
and the National Institutes for Health. Crucially, ESRA stated that 
education research should meet high scientific standards. Before 
ESRA, much of U.S. Department of Education-funded research was 
wasted on fairly unreliable studies that misinformed families and 
educators. 

The most acute problem prior to ESRA was that Department of 
Education-funded studies often made bold causal claims when they 
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used unscientific methods that really could not support those 
claims. Claims of causation (such as stricter teacher licensure rais- 
ing student achievement), were made when the study often showed 
nothing more than a correlation. And in that particular case, it 
turns out that the correlation is not particularly informative about 
the causal effect of teacher licensure on achievement. 

I want to make three main points. The first is that although IES 
has greatly improved education research, vigilance and continued 
improvements are needed. We must continue to raise, not relax, 
scientific standards. 

My second point is that the federal government, universities, and 
philanthropic organizations should share the responsibility for sup- 
porting education research. And my third point is that the research 
functions of the U.S. Department of Education should be the func- 
tions on which people can most easily agree, and this is because 
all markets work better when the people in them are informed, in 
this case parents, students, and educators. 

I think scientific research is one of our best hopes for improving 
American education quickly without our having to spend more 
money. 

So IES has greatly improved education, but now is the time to 
further raise standards, not relax them. I don’t think high scientific 
standards are so ingrained in the education research community 
that IES can afford to take its foot off the gas. 

Since its creation, IES has mainly promoted experimental and 
quasi-experimental methods. These methods tend to produce reli- 
able results as long as they are used properly, and they are not ter- 
ribly difficult to use properly. 

Perhaps something like 10 percent of randomized base studies 
are unreliable, and that number might rise to about 25 percent 
with quasi-experimental studies. That the mistakes are not cor- 
rected by the authors themselves does demonstrate, however, that 
even experimental studies are not dummy-proof. 

Moreover, there are many important questions that cannot be 
answered with experimental studies and the remaining evaluation 
methods require even more expertise to apply. This means that 
IES, if it is to be able to answer all the questions of interest to the 
American people, needs to develop greater expertise. 

Expert review panels are the main means by which IES main- 
tains high standards. While IES reviewing is not yet quite the 
equal of NSF reviewing, in my experience it has made remarkable 
progress, and I would say that the institute is now in a virtuous 
cycle whereby good standards attract good reviewers, and the good 
reviewers attract good proposals. It is a virtuous cycle, but vigi- 
lance is needed because that can easily break down into a vicious 
cycle where poor standards attract poor proposals for research. 

Another thing that IES is doing well but that requires vigilance 
is data collection. IES has traditionally been very strong in col- 
lecting survey-based data, but now most top-notch education re- 
search is migrating away from survey data and towards adminis- 
trative data sets based on schools’ records. This is because most 
scientific methods now require the completeness and the large scale 
of administrative records. 
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Unfortunately, our country is not at the frontier in this. Most 
South American countries and most Northern European countries 
have better administrative data sets than we do. 

This is a problem because researchers tend to migrate towards 
doing research on the things for which there is the best data. For 
instance, right now I could write a much better study of Dutch 
school choice reforms than I could of American school choice re- 
forms. Their data are just better. 

The final thing that IES has done well that Dr. Whitehurst also 
mentioned is really courageously contract for rigorous studies of 
high profile programs. And I would cite as examples the evaluation 
of the D.C. Opportunity Scholarship Program and the evaluation of 
the 21st Century Community Learning Centers. It is not acceptable 
for taxpayers to continue to pay for programs year after year after 
year without having any rigorous evidence on whether the pro- 
grams actually work. 

My second point is that responsibility for education research 
should be shared by the federal government, universities, and phil- 
anthropic organizations. Each one of these entities plays a distinct 
role. 

I have already mentioned that the federal government should col- 
lect data, but it also needs to be a supporter of university-based 
scholars, and I will return to this point. Philanthropic organiza- 
tions also play a key role, but unlike the federal government, they 
should be mainly interested in the evaluation of speculative, inno- 
vative programs, and that is because they are using donors’ money 
to evaluate programs rather than using taxpayers’ money. 

Finally, universities: University-based researchers are primarily 
responsible for developing new scientific methods, validating them, 
and training people to use them. It is essential that these research- 
ers interact with the federal government on a regular basis so that 
cutting edge methods are known by researchers at IES. 

Another important role for university researchers is to work on 
topics that are currently politically unpopular. If they had not been 
doing research on school choice in the 1990s we wouldn’t know very 
much about it today, and they didn’t make very many friends doing 
that research. 

Finally, I said that the research functions of the U.S. Depart- 
ment of Education should be the functions on which people can 
most easily agree. Americans really do disagree on the extent to 
which the federal government should mandate education standards 
and policies, and many Americans believe that it should be families 
who make most of the choices. 

But really no one argues that anyone — the families, educators, or 
policymakers — would be better off if they had less access to reliable 
information, and that means that it is one of our best hopes to im- 
prove education if we use scientific information to spend smarter 
rather than just spending more. Thank you. 

[The statement of Dr. Hoxby follows:] 

Prepared Statement of Dr. Caroline Hoxby, Scott and Donya Bommer 
Professor of Economics, Stanford University 

Me. Chairman and Members of the Committee: My name is Caroline Hoxby. 
I am the Scott and Donya Bommer Professor of Economics at Stanford University 
and the Director of Economics of Education at the National Bureau of Economic Re- 
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search, the nation’s leading nonprofit economic research organization. I served for 
several years as a presidential appointee to the National Board for Education 
Sciences. Over my career, first at Harvard and recently at Stanford, I have con- 
ducted research on a wide array of topics in elementary, secondary, and higher edu- 
cation including class size, charter schools, college tuition, school finance, and bilin- 
gual education. There is a common theme in my research and the research of the 
many Ph.D.s I have trained: we attempt to answer questions in education by apply- 
ing the most reliable, most advanced, most scientific methods to the best available 
data. 

Thank you for the invitation to testify. It is an honor to address you, and I believe 
that today’s topics are absolutely key to improving education in the United States. 

The United States faces a very bleak future if we do not figure out how to quickly 
and continuously improve the education of our population. The American industries 
that are still growing, thriving, and exporting are the industries that are most de- 
pendent on educated workers. If our economy is to grow fast enough to help solve 
our fiscal crisis, we must have a smarter, more productive education sector, not one 
that is simply more costly. 

If this sounds like an insurmountable challenge, it is only because Americans can 
point to so little educational improvement over the past four decades that we, as 
a nation, have begun to believe that very little improvement is possible. Contrast 
this with medicine or almost any other field of applied knowledge. If we were offered 
the choice between a medical procedure that relied on today’s knowledge versus the 
knowledge of 1970, we would — all of us — choose today’s. We would probably be am- 
bivalent about today’s schools versus the schools of 1970. 

The difference between education and medicine is not that improvement is impos- 
sible in education but possible in medicine. It is not that all children are difficult 
to manage and all patients are easy to manage. The difference is that education has 
not, until recently, benefitted from rigorous, scientific research. 

The Education Sciences Reform Act (ESRA) of 2002 greatly transformed education 
research, moving it much closer to the successful models used by the National 
Science Foundation and the National Institutes of Health. ESRA stated unequivo- 
cally that the Institute for Education Sciences (IES) should facilitate research that 
met high, scientific standards in order that it produce reliable results. This was the 
crucial statement. Until ESRA, much of the U.S. Department of Education’s budget 
for research was wasted on studies that were widely recognized to be unreliable. Not 
only was taxpayer money wasted, but the Department unintentionally endorsed and 
promoted poor research methods by funding low-standard studies. 

Prior to ESRA, there were two particularly acute problems with Department of 
Education-funded studies. The first was that they often employed subjective meas- 
ures of what schools did and what students achieved. If a study relies on subjective 
measures, a researcher’s ideology often dictates what the data says. The second and 
more pervasive problem was that Department-funded studies often made bold caus- 
al claims despite the fact that they used methods that could not possibly support 
such claims. Claims of causation — such as “stricter teacher licensure rules raise stu- 
dent achievement” — were made when the study showed nothing more than a cor- 
relation. For instance, in my example, schools with a higher percentage of teachers 
who are licensed are schools that serve students who come from more advantaged 
backgrounds. These students tend to have higher achievement regardless of how 
their teachers are licensed. It turns out that the correlation between teacher licen- 
sure and achievement tells us literally nothing about the causal effect of teacher li- 
censure on achievement. In short, prior to ESRA, Department of Education-funded 
research routinely provided misinformation to American families and schools. 

I support the recommendations that the National Board for Education Sciences 
has already made regarding the reauthorization of ESRA. Those recommendations, 
however, are necessarily detail-oriented. In my remaining time, I wish to provide 
a “big picture” perspective on ESRA, IES, and — more broadly — the role of the fed- 
eral government in education research. 

I have three main points. 

1. IES has greatly improved education research since the enactment of ESRA, but 
vigilance and continued improvements are needed. We cannot afford to relax stand- 
ards now. Rather, even higher scientific standards should be the goal. 

2. The federal government, universities, and philanthropic organizations should 
share the responsibility for supporting education research. This mixed model, some- 
what peculiar to the U.S., is essentially the right model. Each entity plays an impor- 
tant and distinct role. 

3. The data collection and research support functions of the U.S. Department of 
Education should be the functions on which people with diverse political views can 
agree. This is because no market functions better in the absence of information on 
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which parents, students, and schools can make choices. Also, truly scientific re- 
search in education is probably our best hope for improving the skills of Americans 
quickly, with the expenditures we are already making. 

Again, my first point is that IES has greatly improved education but that now 
is the time to further raise, not relax, the scientific standards that are the crucial 
contribution of ESRA. We are not yet in the situation where high, scientific stand- 
ards are so ingrained in the education research community that IES can take its 
“foot off the gas.” Since its creation, IES has consistently promoted scientific meth- 
ods by favoring studies that employ experimental and quasi-experimental methods 
such as randomized controlled trials, randomization built into pilot programs, and 
regression discontinuity. These methods produce reliable results when used prop- 
erly. That is why they are also used in fields such as medicine and social program 
evaluation. Vigilance is needed, however, because even the best experiment is not 
“dummy proof.” IES should continue to raise the bar, insisting on even better train- 
ing in issues like attrition and measurement that arise in experiments. Also, not 
all important questions can be answered with experimental or quasi-experimental 
methods, and IES therefore needs to develop greater expertise in other evaluation 
methods, methods that produce reliable results only when they are applied by re- 
searchers who are very highly trained. 

Expert review panels are the key means by which IES gains access to expert opin- 
ion, maintains high research standards, and improves its own staffs knowledge of 
the latest methods and research. The Department of Education’s expert panels have 
improved greatly since the enactment of ESRA. They now contain a sufficient per- 
centage of well-trained experts that the panel process can be said to, very often but 
not always, fund research that produces reliable results. While IES reviewing is not 
yet equal in quality to the NSF reviewing I have experienced, IES has made re- 
markable progress. The Institute is only able to convene top experts and attract 
high quality proposals because researchers believe that the Department turned the 
comer with ESRA and now promotes scientifically-grounded research. Top experts 
only participate in review processes in which they believe. Top researchers, who can 
devote themselves to issues other than education, only submit proposals to review- 
ers who are expert enough to judge proposals well. In other words, IES is currently 
in a virtuous cycle: higher scientific standards induce participation by more expert 
reviewers. This leads better researchers to submit higher quality proposals, and the 
cycle continues. Vigilance is necessary, however: the virtuous cycle can easily break 
down and become a vicious cycle in which poor standards lead to poor participation, 
at which point the review process attracts only poor proposals. 

Another thing that IES is doing well but that requires vigilance is data collection. 
IES, through its National Center for Education Statistics, has been collecting survey 
data on students and schools for decades. These data tend to be well-respected — 
this is one function of the Department’s research arm that was high quality prior 
to ESRA. However, top-notch education research has migrated away from survey 
data and towards detailed administrative data. About 75 percent of studies pub- 
lished by top applied journals now rely on administrative data — datasets based on 
schools recording what a student does, what teachers and policies and classrooms 
he encounters, and what outcomes he attains, both in the short-term (test scores) 
and long-term (college graduation, earnings, and so on). The reason that research 
is migrating from survey to administrative data is that modern scientific methods 
that produce reliable estimates often require the large scale and completeness of ad- 
ministrative data. While the U.S. continues to have some of the world’s best survey 
data on education, our country has fallen far behind the frontier in administrative 
data on education. Currently, most northern European countries and some South 
American countries have substantially better administrative data than the U.S. 
This matters because top researchers are motivated just as much by the availability 
of data that allow them to write excellent studies as they are motivated by funding. 
Thus, researchers are increasingly drifting away from the analysis of U.S. education 
policies and toward the analysis of other countries’ education policies. To be con- 
crete, I could now write a study of English, Dutch, or Swedish school choice reforms 
using better data than are available to me in the U.S. IES is making valiant efforts, 
which I praise, to create and sponsor stronger administrative databases, but this is 
another area in which continued exertion is needed. Integrating states’ data and 
data from its own agencies (like the National Student Loan system) is probably the 
cheapest and quickest way for IES to improve education research. 

A final thing that IES has done well under ESRA is courageously contract for rig- 
orous studies of high profile programs and programs on which the federal govern- 
ment already spends substantial money. I would cite, as examples, the evaluation 
of the D.C. Opportunity Scholarship Program, the evaluation of the 21st Century 
Community Learning Centers, and the evaluations of Professional Development pro- 
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grams in mathematics and reading. It simply does not make sense for U.S. tax- 
payers to fund programs year after year in the absence of scientific evidence of their 
effects, and findings from such rigorous studies should play an important role in any 
debate about their future. You may have observed that I said these contracts were 
courageous. They were. When one conducts a study using strong, scientific methods, 
one cannot know how it will turn out. It is always possible that some constituency 
will be angered by the results, but — then — that is the entire point of doing research. 
If we could accurately choose education programs simply by knowing “in our hearts” 
that they were right, we would already have very successful schools. 

There are a few areas in which IES has great intentions but is not having the 
effect for which it hopes. The Regional Education Laboratories and the What Works 
Clearinghouse are examples. 

My second point is that support and responsibility for education research should 
be shared by the federal government, universities, and philanthropic organizations. 
In the U.S., we have a successful model in which each of these entities plays an 
important and distinct role. While I would never argue that our model is perfect, 
I am routinely struck by how well it functions when I am abroad and experience 
other countries’ education research. A similar mixed model of support is used for 
medical research. 

The federal government should play a few roles in education research. First, and 
most obviously, it should collect and make available accurate data on all aspects of 
education that can be measured: expenditures, revenues, achievement, personnel, 
curriculum, school policy, and so on. Because there are enormous economies of scale 
and scope in data collection and because cross-state comparisons are so important 
to research, it is important that the federal government and not just state govern- 
ments collect data and make it available in a timely way. 

Second, the federal government should publish descriptive reports on American 
education. The word descriptive is important because such reports are part of the 
government’s duty to disseminate data, rather than a duty to do causal research. 
A report that describes where English Learners enroll is descriptive. This must be 
distinguished from research that attempts to test a causal hypothesis such as 
whether bilingual education raises English Leaners’ achievement. The federal gov- 
ernment is not in a good position to conduct causal research itself. This is because 
such research requires methods that need expert review, and the government can- 
not both convene the reviewers and be the entity that is reviewed. In the same way, 
we would not want an accused person to convene his own jury. A good review proc- 
ess requires independence. 

Third, the federal government should contract for highly reliable evaluations of 
the education programs it supports. These evaluations cost only a small fraction of 
what is spent on the programs themselves. For this small expenditure, a good eval- 
uation can save taxpayers vast amounts of money, either by providing the evidence 
that improves a partially-successful program or by providing the evidence that gives 
Congress the grounds for abolishing an unsuccessful program. The federal govern- 
ment should be prepared to fund evaluations of its programs with little financial 
help from universities or philanthropies. This is because the goal of such evalua- 
tions is not to be innovative or to explore new questions. The goal is to produce clear 
answers to well-specified questions regarding established programs. The ideal eval- 
uation should employ methods that are well-validated that the evaluation is boring 
in every way except for the results. Fortunately, in the U.S., we have active com- 
petition for such contracts among a good number of organizations: Mathematica, 
Abt, Rand, Westat, AIR, MDRC, and so on. 

Fourth, the federal government should share in the support of (but not be the ex- 
clusive supporter of) research by university-based and similar scholars. These are 
the people who develop new methods, who ask questions that are still somewhat 
speculative, and who conduct “basic research” in education. I will return to this 
point. 

Philanthropic institutions also play a vital role in education research. In some 
ways, their role parallels the federal role except that philanthropies should focus 
more on trial programs that are innovative and less on established programs funded 
by the government. This is because the government uses money that taxpayers are 
obliged to pay while philanthropic organizations use money that their donors freely 
give. If a philanthropy spends money on a speculative educational program that 
does not succeed, the consequences fall on its donors — people who are affluent 
enough to accept this risk in return for the prospect of developing exciting new pro- 
grams that benefit society. Philanthropies can obtain reliable evaluations by con- 
tracting with the same organizations that contract with the federal government. 
And, like the government, philanthropies should share in supporting research by 
university-based and similar scholars. 
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Let me now turn to the role of university-based researchers. As I mentioned, uni- 
versity-based researchers are primarily responsible not only for developing new and 
more scientific methods of evaluation, but also testing them, validating them in an 
array of applications, and training people to use them. For instance, university re- 
searchers developed the cutting-edge methods to deal with attrition and non-compli- 
ance in randomized controlled trials. They also developed the quasi-experimental 
methods that are currently the workhorses of evaluation. In addition, university- 
based researchers are almost entirely responsible for conducting basic research — re- 
search that has no immediate policy relevance but that provides fundamental infor- 
mation on which policies should be ultimately based. For example, I study peer ef- 
fects — how students’ achievement is affected by the other students who share the 
classroom with them. This basic research is a fundamental that we need to evaluate 
policies like school choice that affect which students are in each school. Another 
good example of basic research is the recent spate of studies that show (a) that dif- 
ferent teachers have very different effects on achievement and (b) that a teacher’s 
effect is not related to her credentials. This basic research is a fundamental we need 
for thinking about teacher pay incentives, teacher training, teacher tenure, and poli- 
cies that affect which teachers end up in which schools. Finally, university research- 
ers should be primarily responsible for investigating educational programs that are 
speculative, still under development, or implemented on a purely trial basis. Univer- 
sity researchers must also do the uncomfortable work of analyzing programs that 
are currently unpopular with the administration and/or philanthropies. As an exam- 
ple of a purely trial program designed and investigated by university researchers, 
I would point to the recent study that shows that students are more likely to enroll 
in college if their family can automatically file the Free Application for Federal Stu- 
dent Aid when it files its taxes. As an example of unpopular research, I would point 
to studies of school choice from the 1990s. Researchers who worked on such topics 
did not win many friends in the education establishment, but we are now glad that 
the studies exist because they inform us about how to structure choice policies. 

I have said that the federal government and philanthropies should share in the 
support of university-based education research. Why? If the government and philan- 
thropies do not have “skin in the game”, they will not attract university researchers 
to study the policies or develop the methods that are important to them (the govern- 
ment and philanthropies). They will not attract top experts to review the contract- 
based studies they support. They will not learn about cutting-edge research and cut- 
ting-edge methods in real time. It is the nature of cutting-edge work that you cannot 
learn about it just by reading an article after the fact. You need to interact with 
researchers — ask them questions, pose alternatives. 

Universities themselves should also share in supporting education research. Why? 
If we want university-based researchers to invent better methods and conduct basic 
research, they need to be rewarded for these activities. No one is better at gener- 
ating these rewards than universities themselves. This is because universities’ con- 
stituents give them incentives to create knowledge that is original and a public 
good, as all basic research is. 

By sharing in the support for educational research, the federal government, uni- 
versities, and philanthropists also share in setting the research agenda. This is a 
good thing. Innovation never benefits from one entity having a monopoly on what 
questions are interesting. 

My third and final point is brief. The data collection and research support func- 
tions of the U.S. Department of Education should be the functions on which people 
can most easily agree. Americans tend to disagree on the degree to which the fed- 
eral government should mandate educational standards and impose policies on 
schools. Many Americans believe that families and local communities should make 
education choices for themselves. But, it is hard to argue that anyone — families, 
communities, schools, or federal policy makers — will make better choices if they 
have less access to reliable information. As I stated at the outset, Americans badly 
need to be better educated — and soon — because our economic growth and well-being 
depend on this. I truly believe that our best hope is to improve education by spend- 
ing smarter — using scientific methods to identify which programs and policies are 
effective and which are counterproductive or just a waste of money. 


Chairman Hunter. Thanks. I think we can all agree, too, that 
5 minutes really is not that long to talk, is it? Not very long. 

Mr. Fleischman? 
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STATEMENT OF MR. STEVE FLEISCHMAN, DEPUTY EXECUTIVE 
OFFICER, EDUCATION NORTHWEST 

Mr. Fleischman. Chairman Hunter, Mr. Holt, members of the 
subcommittee, thank you for this opportunity to offer testimony. I 
think that what I say will continue in the theme of trying to pro- 
vide better evidence for better decisions. 

I am Steve Fleischman. I am the deputy executive officer of Edu- 
cation Northwest. I have been involved in the promotion of evi- 
dence-based education for more than 15 years. 

I believe, however, that my most important qualification for of- 
fering testimony today is that I am a former middle and high 
school teacher. When I entered teaching as a second profession in 
the mid-1980s there was almost no evidence that I could find to 
help me manage my class better, teach my history lessons more ef- 
fectively, improve the writing skills of my students, on and on. We 
have come a long way since then, but not far enough. 

Before becoming a teacher I was a business person, and I often 
think in market terms. To me, the challenge in building an effec- 
tive education research enterprise is to create a market that has 
mechanisms to supply high quality research, create demand for it, 
and ease its use. 

Peter Drucker often observed that there is no business without 
a customer. Simply put, I believe that we will not have an edu- 
cation system in which reliable evidence is widely used to drive de- 
cision making unless and until we provide educators the research 
they want and need. 

Recent studies on research use by educators, including one con- 
ducted by my own organization, document this research-to-use gap. 
Three findings from our study, however, suggest important prin- 
ciples to narrow this gap. 

One: Research should be contextualized. The observation that all 
politics is local has its equivalent in the observation that all re- 
search is local. That is, participants in our study expressed a 
strong preference for research evidence that is linked to local con- 
tacts. 

Two: Research should be easy to read, absorb, and apply. Partici- 
pants expressed preferences in how studies should be presented, in- 
cluding the report should be brief and written in nontechnical lan- 
guage. 

Three: Research often requires translation and transmission by 
intermediaries. Intermediaries were identified by the participants 
as unbiased organizations and individuals that can help locate, 
sort, and prioritize the available evidence. 

IES has taken significant strides in promoting an increase in the 
amount of rigorous evidence available to educators. As well, re- 
gional educational labs and the What Works Clearinghouse have 
begun to move forward the relevance and usefulness agenda. 

Some of the promising practices and developments initiated by 
IES working with other program offices of the Department of Edu- 
cation include the production of so-called practice guides; the hold- 
ing of REL bridge events; the Ask A REL information services; co- 
ordination across the Department of Education in fields such as re- 
search, development, and technical assistance projects; and an in- 
creased focus in meeting the real-world improvement needs of edu- 
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cation stakeholders that I think is exemplified in the new REL 
competition statement of work. 

My suggestions regarding how ESRA can be improved in the 
next reauthorization result from many conversations, including 
those held by members of Knowledge Alliance, a trade association 
of leading education R&D organizations. These recommendations 
are: One, engage consumers. The most powerful way to increase re- 
search use is to engage the prospective consumers of evidence in 
defining the practical problems that should be analyzed, designing 
the modes in which findings will be presented, and supporting 
ways for the evidence to be applied effectively in the field. 

Two: Pay attention to implementation. Research consistently 
demonstrates that even the best programs fail to provide their in- 
tended benefits if poorly implemented. Therefore, greater focus 
should be devoted to learning more about how strong programs and 
practices can be implemented well. 

Three: Support intermediaries. As noted above, research con- 
sumers often turn to intermediaries who serve as trusted sources 
that help sort through the evidence. Many of these trusted sources 
represent projects and individuals either directly support through 
current federal research, development, and technical assistance in- 
frastructure or interact with this infrastructure. 

Fourth and finally: Promote the coordination of U.S. Department 
of Education program offices. Taking the point of view of con- 
sumers of evidence, education stakeholders should have a much 
clearer idea of who to contact and what services are available to 
meet their evidence needs. 

I believe that when Congress passed ESRA and created IES it 
had a vision that science, properly conducted and effectively ap- 
plied, could be a significant engine in improving education in this 
country. As Mr. Holt has written and argued, recent history dem- 
onstrates that investments in R&D can drive the economy forward. 
Yet, the Department of Education spends less than 1 percent of its 
budget on R&D, one of the smallest investments of any federal 
agency. 

Ongoing federal investment in education research enterprise will 
be required if we are to achieve the promise that all students will 
receive a quality education that prepares them for fulfilling lives 
as contributing citizens in our society. 

Thank you. 

[The statement of Dr. Fleischman follows:] 

Prepared Statement of Steve Fleischman, Deputy Executive Officer, Edu- 
cation Northwest; Director, Regional Educational Laboratory Northwest 

Chairman Hunter, Ranking Member Kildee, and Members of the Sub- 
committee: Thank you for this opportunity to offer testimony as you consider how 
education research can help to promote the identification and use of effective pro- 
grams to support students and teachers. 

I am Steve Fleischman, the deputy executive officer of Education Northwest. We 
are a nonprofit organization created in Oregon more than 45 years ago to apply re- 
search to improve education in the Northwest, and across the country. Some of the 
projects that we conduct on behalf of the U.S. Department of Education, and which 
provide part of the experience base for my testimony include the Regional Edu- 
cational Laboratory (REL) Northwest, Northwest Regional Comprehensive Center, 
and the Region X Equity Assistance Center. 

I have been involved in the promotion of evidence-based education for more than 
15 years. In the last decade, with different organizations, I have participated in a 
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variety of U.S. Department of Education projects to provide educators better evi- 
dence, including serving as the first communications director of the What Works 
Clearinghouse, director of a project to provide education decisionmakers with con- 
sumer reporting on the quality and effectiveness of school reform models, and senior 
leader of the Doing What Works project. Currently, I serve as director of REL 
Northwest. These and other projects in which I have been engaged have given me 
insight into the need for better evidence in education that helps identify and imple- 
ment effective programs and practices. This need led to the passage of the Edu- 
cation Sciences Reform Act (ESRA) in 2002, and the creation of the Institute of Edu- 
cation Sciences (IES). 

I believe, however, that my most important qualification for offering testimony is 
that I am a former middle and high school teacher. When I entered teaching as a 
second profession in the mid-1980s I did what most other new teachers do: scramble 
desperately for any support to help do my job. One of the places I turned to was 
research literature on best practices. There was almost no evidence I could find to 
help me manage my class better, teach my history lessons more effectively, improve 
the writing skills of my students, or do any of the other things I needed to do to 
be a good teacher. This experience has been the single most important one in help- 
ing to guide my actions for the past 15 years, as I’ve been increasingly involved in 
the education research enterprise. Although the situation is much better today than 
a quarter of a century ago, we have a long way to go before education research ful- 
fills its promise as an engine of educational improvement. 

Before going further in my testimony on the topic, I would like to clarify how I 
will use the term “education research.” My experience is that when making deci- 
sions, educators in the field are focused on “evidence use” which can include formal 
research, program evaluations, reviews of bodies of research, and various data. That 
is, educators turn to many sources of “evidence” when searching for guidance on pol- 
icy and practice, formal research being only one of them. In this testimony, I will 
use this more expansive conception of “education research” that encompasses the 
sources just mentioned. 

Start with the consumer 

Before becoming a teacher, I was a business person, and I often think in market 
terms. To me, the challenge in building an effective education research enterprise 
is to create a market that has mechanisms to supply high quality research, create 
demand for it, and ease its use. Peter Drucker, the revered management thinker, 
often observed that there is no business without a customer. Simply put, I believe 
that we will not have an education system in which reliable evidence is widely used 
to drive decision making unless and until we provide educators the research that 
they want and need. 

The past decade has seen advances in increasing the supply of rigorous education 
research as well as some closing of the “research-to-use” gap. In my testimony I will 
suggest ways that federal investments and action can help to further close this gap. 

Recent studies on research use by educators point to this ongoing challenge. For 
example, in a 2009 study that my organization and others conducted for the William 
T. Grant Foundation, a wide ranging group of education practitioners and policy- 
makers observed that: 

• There is a gulf between research design and real-world practice, which often re- 
sults in findings that have limited applicability. 

• They are challenged to apply research because of their own lack of knowledge 
and skills in acquiring and interpreting research. 

• Numerous obstacles exist to research use, including “time constraints, the vol- 
ume of research evidence available, the format in which it is presented, and the dif- 
ficulty in applying research to their own situations.” 

• They are often skeptical about research and concerned that it is conducted and 
reported for ulterior motives or can be shaped to “say anything.” 

• Research is often not timely. 

Most troubling is the fact that none of the study participants could identify any 
“breakthrough” research or “cite any findings that they feel had a dramatic effect 
on practice or policy.” 

Principles for increased research use 

Our study cited above and others point to current gaps, but also to ways to im- 
prove the connection between research and practice. Three findings from our study 
suggest important principles to narrow the “research-to-use” gap: 

1. Research should be contextualized. The observation that “all politics is local,” 
has its education research equivalent, in which “all research is local.” Participants 
in our study expressed a strong preference for research evidence that is linked to 
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local contexts. Thus, for research to be seen as useful and to be used, it must be 
contextualized. One way to accomplish this is to involve education research con- 
sumers in studies from the very beginning: in setting the questions, designing the 
studies, and writing reports that answer questions of local interest. 

2. Research should be easy to read, absorb, and apply. Participants expressed 
preferences in how studies should be presented, including that reports should be 
brief and written in non-technical language. This principle suggests that much more 
attention needs to be paid to communicating research effectively. Otherwise, poten- 
tially important research findings might not be read at all. 

3. Research often requires “translation” and “transmission” by intermediaries. 
Intermediaries were identified by the participants as “unbiased organizations and 
individuals that can help locate, sort, and prioritize the available research.” Among 
examples identified by participants were “research institutions, professional associa- 
tions, partners, coalitions, peers, networks, and constituents.” A key implication is 
that it is important to find ways to strengthen the role of intermediaries by making 
sure they have the knowledge, skills, and resources to play this important role. 

The IES track record on promoting research use 

Since the passage of ESRA nearly a decade ago, IES has taken significant strides 
in promoting an increase in the amount of rigorous evidence available to education 
decision makers. It has improved the quality of quantitative research and data 
through various mechanisms including grant competitions, sponsored research, and 
the operation of the National Center for Education Statistics, Regional Educational 
Laboratory (REL) system, and the What Works Clearinghouse (WWC). While some 
of these mechanisms have focused largely on increasing research and data rigor oth- 
ers, particularly the RELs and the WWC, have begun to move forward the relevance 
and usefulness agenda necessary to meet consumer needs and desires for evidence. 

Some of the promising practices and developments initiated by IES, working with 
other program offices of the Department of Education, include: 

• The production of Practice Guides. These guides, currently numbering 14 and 
largely produced by the WWC, offer practical recommendations based on the best 
available evidence. Developed by panels of nationally recognized researchers and 
practitioners, they offer actionable recommendations, strategies for overcoming po- 
tential practice roadblocks, and an indication of the strength of evidence supporting 
each recommendation. Topics range from turning around low-performing schools 
and reducing high school dropouts, to using data to support instructional decision 
making and structuring out-of-school time to improve academic achievement. 

• The holding of REL Bridge Events. These are in-person or webinar events held 
for education stakeholders across the nation by the 10 RELs to share and discuss 
the recommendations of the Practice Guides and other rigorous and relevant evi- 
dence. The events have proven to be highly popular and represent a key mechanism 
to link educators to the “best available” research-based guidance on critical topics 
of regional or local interest. 

• Ask A REL information services. Every REL offers this free service that allows 
education stakeholders to call or e-mail with their questions of practice. These ques- 
tions are posed by state officials, school board members, superintendents, principals, 
teachers, parents, and others seeking to find our “what the research says” on par- 
ticular topics. The requests, which are turned around quickly, often result in re- 
search literature reviews that are then shared with other stakeholders. 

• Coordination across the U.S. Department of Education research, development, 
and technical assistance infrastructure. Centers and projects sponsored by various 
Department program offices have come together more regularly than in the past to 
hold joint activities that provide stakeholders needed information. One example was 
a series of regional events on School Improvement Grants (SIG) jointly sponsored 
by the RELs and Comprehensive Centers this past year. In another recent example, 
REL Northwest worked together recently with two regional comprehensive centers 
and the Center on Innovation and Improvement to bring together state officials and 
leaders from rural SIG schools in five states to learn about effective practices to 
turn around their schools. 

• An increased focus of the REL system on meeting the improvement needs of 
education stakeholders. In a highly encouraging development, the current IES com- 
petition for new REL contractors that will launch a new five-year cycle of REL activ- 
ity beginning in January 2012 is focused on the creation of research and data-use 
partnerships with educators and policymakers in the field. These so-called “research 
alliances” will be long-lasting, help to set the research agendas for the RELs so that 
they concentrate on real world “problems of practice,” and provide capacity building 
so that alliance partners are increasingly able to conduct their own research and 
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data-analysis projects. Without sacrificing rigor, these alliances will go a long way 
in deeply engaging consumers of research in its production and use. 

Considerations for ESRA reauthorization 

Discussions in the education research community regarding how ESRA can be im- 
proved in its next reauthorization have been ongoing in the field for several years. 
For example, Knowledge Alliance, a trade association of leading education research 
and development (R&D) organizations that I currently chair, has engaged its mem- 
bers and experts in the field in this discussion. As well, my own organization’s 
Board of Directors composed of nearly 30 education stakeholders across the states 
of Alaska, Idaho, Montana, Oregon, and Washington, has been discussing this issue 
over the past two years. The considerations below are suggestions that have arisen 
from these conversations. As Congress considers reauthorization of ESRA, I rec- 
ommend that you keep in mind the following goals which might result in new mech- 
anisms and practices or the strengthening of current ones to better connect evidence 
and practice: 

• Engage consumers. The most powerful way to increase research use is to engage 
the prospective consumers of evidence in defining the practical problems that should 
be analyzed, designing the modes in which findings will be presented, and sup- 
porting ways for the evidence to be applied effectively in the field. This should in- 
clude building consumer capacity to find, judge, and apply evidence that is provided 
at the federal level and beyond. Key consumers on which to focus capacity building 
efforts might be state education agency and local district staff who lead research 
and data analysis tasks. Finally, this effort might include studies and other efforts 
to determine how to better serve education consumers’ evidence needs. 

• Pay attention to implementation. The identification and sharing of effective pro- 
grams and practices represents only part of an effort to promote an evidence-based 
education system. Research consistently demonstrates that even the best programs 
fail to provide their intended benefits if poorly implemented. Therefore, greater 
focus should be devoted to learning more about how strong programs and practices 
can be implemented well. 

• Support intermediaries. As noted above, research consumers often turn to inter- 
mediaries who serve as trusted sources that help sort through the evidence to find 
that which is most relevant for consumer decision making needs. Many of these 
trusted sources represent projects and individuals either directly supported through 
the current federal research, development, and technical assistance infrastructure 
or that interact with this infrastructure. Examples of the latter are associations of 
state education officials, school boards, administrators, principals, teachers, and 
education journalists. These intermediary organizations must be engaged and sup- 
ported systematically if we are to improve the connection between research and 
practice. 

• Promote the coordination of U.S. Department of Education program offices. 
There are notable examples of how program offices such as IES, the Offices of Ele- 
mentary and Secondary Education, Innovation and Improvement, Special Education 
Programs, and others work together to promote evidence use. However, there is 
much more that can be done to promote this coordination. Taking the point of view 
of consumers, education stakeholders should have a much clearer idea of who to 
contact and what services are available to meet their evidence needs. Applying this 
customer-based approach would require the U.S. Department of Education to struc- 
ture its information and support activities in a more coordinated way to promote 
an evidence-based system. 

Federal investments in education research can pay dividends 

This testimony has focused largely on the supply side of the research use equa- 
tion, in the hopes that if research can be made more timely, relevant, and useful, 
it will more likely factor into decision making. However, there other aspects of “mar- 
ket building” that I have only mentioned briefly in this testimony that require a fed- 
eral role. For example, ongoing federal communication regarding the importance of 
evidence use sends a powerful signal in the system to promote its use. Emphasis 
in federal education technical assistance that increases the capacity and support 
provided for evidence use increases the likelihood that research and data will be 
used effectively. 

In the early 1950s, parents in this country had to worry about their child con- 
tracting Polio, the dreaded disease of the day. In 1952, the year before I was born 
more than 3,000 children, a record number, died from the disease. Today, thanks 
to significant investments in scientific research and effective public health cam- 
paigns, Polio no longer exists in this country. However, what does still exist in 
America are far too many crippling conditions such as students who cannot read by 
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grade three, drop out before completing high school, or reach college unprepared for 
success. Like Polio, these conditions demand a substantial investment in research 
and then translation of that research into practical action. 

I believe that when Congress passed ESRA and created the Institute of Education 
Sciences, it had a vision that science, properly conducted and effectively applied, 
could be a significant engine in improving education in this country. Further, as Mr. 
Holt, a member of this subcommittee, has argued, recent history “demonstrates that 
investments in R&D can drive the economy forward.” Yet, the Department of Edu- 
cation spends less than 1 percent of its budget on R&D, one of the smallest invest- 
ments of any federal agency. 

Ongoing federal investment in the education research enterprise will be required 
if we are to achieve the promise that all students will receive a quality education 
that prepares them for fulfilling lives as contributing citizens in our society. 

In closing, thank you for this opportunity to offer testimony today. 
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Chairman Hunter. Thank you. 

I would now like to recognize Dr. Smith for 5 minutes. 

STATEMENT OF DR. ERIC SMITH, FORMER FLORIDA COMMIS- 
SIONER OF EDUCATION, FLORIDA DEPARTMENT OF EDU- 
CATION 

Mr. Smith. Thank you very much. Mr. Chairman, I’m honored to 
be before all of you this morning on what I consider to be an ex- 
traordinarily important issue. I will first state that I am not a re- 
searcher, but I am a consumer of research. 

I have been a classroom teacher, school administrator, district 
administrator, and B.S. in Newport News, Virginia, and a state 
commissioner, and have had a strong belief in the work of people 
like Ron Edmonds, and others, that give this notion that when 
leadership chooses to make a difference with the outcome of chil- 
dren, when they are committed to making that difference they will 
do so wherever it is important to them to do so. 

It is the fabric of the basis of accountability. That accountability 
hinges on the ability to make smart decisions about what you do 
in schools and classrooms and districts and states. To call yourself 
a reformer requires that you have the ability to move systems in 
a way that brings with it progress and improves student achieve- 
ment and student outcomes. 

Over the years I have had the opportunity to not only work as 
a consumer, but also been asked to serve as the chairman of the 
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Title 1 Review Committee. For a good number of years, I worked 
with Russ Whitehurst and others, and that was an exciting time 
for me. 

That was a time when I did see, in my view, dramatic improve- 
ment in the way we approach the question of research, what is im- 
portant for a high school principal to do in their schools? What is 
important for a district superintendent that is under pressure on 
outcome and achievement to modify and change the way they de- 
liver reading and mathematics? 

One of my questions that I had of Russ one day late in the after- 
noon was, in this great nation, can’t we tell our educators what the 
most effective method of teaching mathematics is? Should it still be 
a question out there for those that have to and are expected to de- 
liver every day? 

So it has been excited to serve in those capacities. Brilliant peo- 
ple and great passion around trying to find the right answers. 

For the consumer there are two big questions, though, that get 
divided — it is kind of inside baseball to me — divided between how 
research is conducted and the quality of the research, and then 
how that research is disseminated. To the superintendent, to the 
building principal, the classroom teacher, those lines are blurred, 
and it is kind of an inside discussion about how that works. All 
they know is they want an answer and they can’t find it. 

We have seen dramatic, dramatic improvement, and I would, 
part of my recommendation is to continue to fund the kind of re- 
search that has been done. But often it comes out rigid, it comes 
out, in order to get it right, to have all the controls in place, it be- 
comes so different from the real world it becomes hard for a practi- 
tioner to put it into place. 

What do I do with this? Those aren’t my classrooms. That is not 
my district. 

And so that translation, whether it be fault of the research and 
the way it is conducted or it is a failure of the translation and the 
dissemination of the research, there is still — we are better; we have 
farther to go. There is more work to be done, and those lines are 
blurred. 

So I would move, in my remaining 1 minute and 27 seconds, to 
talk specifically about some recommendations. One is that — I will 
start with my second recommendation in my written remarks 
first — is that I do think we need to be very thoughtful, very wise 
about broadening our scope on how we might gain information; I 
wouldn’t say even conduct research, but gain information about ef- 
fective practices. I think we can make them more relevant, more 
timely, and more cost effective by broadening our views. 

There are places and things that need to be under a rigorous sci- 
entific model and approach, but as I stated in my comments about 
the application of PSAT, it came from a relational table on the back 
of a document produced by — that I get annually. It was translating 
that that makes a difference, so, and as a result we translated 
data, information about a product, and we made lives different for 
tens of thousands of children across America — dare I say, hundreds 
of thousands of children across America. So there are other ways 
of knowing what is important. 
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On that note, on the research side it is important that practi- 
tioners, the consumers, help to drive the problems that they have 
today and the conditions that exist today, and that researchers 
help inform the best way to get at that answer but look at it from 
a broad view. 

Second, on dissemination, it is hard to get a superintendent or 
a principal’s time, or a commissioner’s time. They are swamped. 
And so the ability to go through existing channels where practi- 
tioners will be there and show up, if it is not important to be on 
the keynote panel it probably isn’t important to those out in the 
field. So the dissemination process, how that is done, is critically 
important. 

And I would just finally say is that what this committee, the 
question this committee is asking, is everything. It is about reau- 
thorization. It is about accountability. It is about school reform. It 
is about our nation’s future. 

Our ability to know what works and how to get it in the class- 
rooms is of critical importance today for children sitting in class- 
rooms at this moment. Thank you very much. 

[The statement of Dr. Smith follows:] 

Prepared Statement of Dr. Eric J. Smith, Former Florida Commissioner of 
Education, Florida Department of Education 

I appreciate the opportunity to address this committee about a topic I find to be 
extraordinarily important to our nation’s academic progress; research on the tools 
and strategies that we give our teachers to use in the classroom. I am speaking to 
you not as a researcher but as a consumer of research on educational strategies, 
tools and practices. In my career I have had the honor of serving 7 years as a class- 
room teacher, 8 years as a high school administrator, 17 years as a superintendent 
and 4 years as a state commissioner of education. Throughout my career, in each 
of these positions, I have been constantly searching for tools, strategies and prac- 
tices that had some independent evidence that if properly used would result in posi- 
tive outcomes for children. Said another way, I have always been searching for those 
tools and techniques that are unique, that can be used in a single classroom or used 
on a large scale and will generally result in positive achievement gains for children. 
To be blunt it has been a frustrating search. There are numerous approaches to 
choose from and as a consumer you will always be told that educational practices 
and tools are aligned to your standards, are research based and you will always be 
shown data that is intended to demonstrate that an instructional approach is ex- 
tremely successful in raising achievement levels. Unfortunately there is still far too 
little independent research or information on the impact of various approaches to 
student learning. 

My interest is in trying to find better information on the effectiveness of edu- 
cational tools and approaches to help teachers, administrators and governing bodies 
to make more informed strategic decisions on how to improve student achievement. 
The question of effectiveness and impact is central to discussions of accountability, 
and should be part of the foundation in the development of reform strategies. School 
reform and accountability have as a premise that leadership can shape and control 
for academic outcomes by thoughtful strategic planning and execution. There is also 
an implicit assumption that the needs of individual children can be addressed 
through the careful planning of practices and strategies as well. Common variables 
include conditions of time, resources and quality instruction using high quality ma- 
terials. The primary classroom materials chosen and given to the teacher to deliver 
the level of instruction required is an essential component. An example of con- 
necting instructional strategy to the needs of individual students is found in the 
emerging development of adaptive testing. Adaptive testing is showing great prom- 
ise in helping educators to be much more student centric in the delivery of instruc- 
tion and in meeting the individual needs of students. Reform strategies such as 
these should be built around the type of work that is to be done in the classroom. 
Such strategies should be framed by the selection of classroom practices and selec- 
tion of primary and supporting classroom materials. Those that make the decisions 
about classroom instructional practices and materials should be held accountable for 
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their decisions. I have been in classrooms where a school or district has selected an 
instructional strategy with supportive tools and you will see teachers who have so 
little confidence in the approach, that they secretively have hoards of other mate- 
rials to do the job. The quality of instructional tools and approaches matters to 
teachers and matters to students. Some help, some don’t offer much and it can be 
assumed that some may do harm. 

So my interest in the question of what classroom practices and tools are effective 
resulted in me being selected to chair the Title 1 Independent Review Panel. It was 
an extraordinary experience. My colleagues on the panel were both brilliant and 
passionate about the issue of instructional improvement. I credit the work of Russ 
Whitehurst and others for pioneering a new way to look at the process of edu- 
cational research. It was bold and aggressive and had the intent to base findings 
on a scientifically rigorous research methodology. As superintendent in Charlotte, 
our children benefited from much of these early efforts to redefine the research. In 
Charlotte we had no district wide strategy for reading instruction. You could go into 
an elementary school and reading would be taught differently at the opposite ends 
of a hallway. Strategically we needed to go to a district adoption so all teachers 
could be supported through professional development and adequate materials. But 
the question was: what approach would be most helpful for students? National re- 
search helped us make that decision and it was the right decision, reading achieve- 
ment went up dramatically. Down the road in Florida, at about the same time, the 
entire state was making decisions about reading. Those decisions were also being 
informed by quality research and the results over the last decade have also been 
extraordinary. 

But often a strict application of scientific research has significant challenges; the 
selection of the control group can be difficult if you are fairly certain that the inter- 
vention will be beneficial. There is also difficulty in maintaining the fidelity of the 
experimental group in a real situation and the process is slow and expensive. The 
instructional strategies will ultimately be used in states, schools and districts that 
don’t have strict and rigid structures, kids come and go in classrooms as do teach- 
ers, schedules get interrupted, materials sometimes are in short supply and profes- 
sional development can be delivered with varying quality. As a result, the nature 
of the research often fails to mirror reality. The research methodology has the tend- 
ency to be cumbersome in its implementation and lead to findings that are rigid and 
artificial. As a result, the research has limited relevance to the real conditions found 
in schools and classrooms. 

Research that is available is also proving difficult to disseminate and get in to 
the hands of those who have the responsibility for making educational decisions. 
The regional labs are of widely different quality and unfortunately are not the “go 
to” place for information on meaningful research. Some of the labs do very good 
work but the quality and reputation varies, and as a result, they don’t form a net- 
work of dissemination that provides national coverage. The What Works Clearing- 
house is making good strides in dissemination, but is limited on bridging the re- 
search to application challenge, research findings are slow to become available and 
because of the nature of the research often lacks application in real situations. 

My recommendations going forward are three fold; 1. continue to support inde- 
pendent research on the quality of educational strategies, tools, and practices, 2. de- 
velop new methods to gain insight into the effectiveness of educational strategies, 
tools, and practices and 3. expand and create new channels for the dissemination 
of educational research. 

My first point; the need for continued support for education research is critical 
because it is so central to all discussions of accountability and reform. I often say 
that schools don’t fail, districts fail. The reason for that belief is that most of the 
important decisions relating to how a school operates are made at the district level; 
leadership, hours of instruction, calendar, staffing restraints and yes, selection of in- 
structional tools and practices. The ability of a district to make sound strategic deci- 
sions about their selection of tools and practices is dependent on quality and timely 
information regarding the impact of the tools and practices. That should not be done 
district by district. States and the Federal Government have a responsibility to sup- 
port independent research on the educational effectiveness of tools and practices. 
The research should be led in large part by practitioners, answering questions that 
are timely and relevant to their work with children. 

Regarding my second point, in my testimony I have cited two examples where 
children benefited by making strategic decisions that were informed by quality re- 
search. I would also share that I have used other methods of gaining insight into 
the quality and effectiveness of educational practice that weren’t based on rigorous 
scientific methods and proved to be very timely, cost effective and also resulted in 
significant benefit for students. I would give you one example. In Charlotte, one of 
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my staff noted that the correlation between a student’s PSAT scores and AP per- 
formance could be built into a program, and rosters of students that had good poten- 
tial for success in AP could be generated. These simple correlation tables provided 
valuable insight into the use of the PSAT. The impact of knowing the correlation 
information and being able to apply it resulted in significant increases in college 
level high school work that was being offered to students and resulted in a signifi- 
cant increase in overall college readiness for the students in Charlotte. A second ex- 
ample is from my work as Commissioner in Florida. As Commissioner, I was able 
to develop plans that will expand our statewide data base to include the primary 
instructional practices and tools used in each classroom. The intent was that we 
could develop relational information between instructional practices and tools and 
student achievement in a variety of different school settings. These findings would 
be made available to districts for use in their strategic planning process. 

Finally, there needs to be a stronger link between educational research and real 
world application. If there is a judgment about strictly designed research versus real 
world conditions of application, the call should favor the real world conditions in 
every instance. Information that is disseminated needs to be timely, addressing 
challenges the field has today not yesterday. It needs to address broadly defined 
challenges, the big questions, not narrowly defined questions that have little rel- 
evance. And dissemination needs to utilize existing organizations to communicate 
findings such as The Council of Great City Schools, CCSSO, Chiefs For Change, 
AASA and ASCD to name a few. If the research findings are not of interest to these 
organizations, they won’t be of interest to their members either and dissemination 
will fail, fail because the research is not important. 

This committee is addressing an issue of great national importance, important to 
our country and also important for our children. I commend you for your work. 


Chairman Hunter. Thank you. 

I thank the panel, once again, for your testimony. 

I would like to start out by saying that it is interesting — I spent 
some time in the Marine Corps, so if you noticed, in the U.S. mili- 
tary there are four different uniforms, and when you go to combat 
there are four different uniforms — two or three, because the Navy 
wears the Marine Corps uniform now. But the problem is, if you 
look at it scientifically — and you should be able to using different 
light spectrums and so forth, and matching up the uniform with 
the surroundings and the environment in which you are fighting — 
there has got to be one good uniform. 

There is obviously, if you test these uniforms using different 
spectrums of light and so forth, there is one uniform that protects 
the wearer better than any of the other uniforms do. But if that 
was true then we would all just have one uniform. We wouldn’t 
have a Marine uniform, and an Army uniform, and an Air Force 
uniform, but that is what we have. And you would think — it is kind 
of sad, if we can’t do it at that level what makes us think we can 
take best practices and scientific research and datum and use it at 
this level. 

And secondly, there seems to be some disparity between the abil- 
ity to get the best practices and things that work and the best cur- 
riculums for teaching teachers that then transfer to teaching stu- 
dents, and then the implementation — there seems to be a dis- 
connect. I don’t necessarily think we are talking about that dis- 
connect today or the implementation of research and your findings, 
but that has got to be, that is, that gap has to be bridged at some 
point, and that is going to be fairly difficult to do, I think. 

First, Dr. Whitehurst, you say it is a mistake for Congress to dic- 
tate how schools — how states and school districts should use find- 
ings from research. Can you provide some examples of this? 
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Mr. Whitehurst. Yes, Mr. Chairman. I mentioned Reading First 
as an example of the federal government specifying at a fine level 
of detail how reading should be taught. There is no evidence indi- 
cating that kids are reading better as a result. 

We have in current federal policies and Race to the Top specifica- 
tions an indication that there are four ways that a failing school 
should be turned around. How do we know that there aren’t nine 
ways, or seven ways, or six ways? 

So to try to get down to the operational level, in terms of how 
a teacher should do his job, or how a district superintendent should 
do his job, through legislation seems to me to be a mistake. And 
if you go through the current version of the Elementary and Sec- 
ondary Education Act you will find almost every section of the bill 
dictates that practice be based on research findings. 

Often, the research findings aren’t there. I remember when Dr. 
Smith pulled me aside and said, “Well, what math curriculum 
should I be using?” And I said, “I don’t know.” And he said, “How 
could you be requiring me to meet the mandates of No Child Left 
Behind to use the scientifically-based research and there is no re- 
search to tell me how to do that?” 

So, you know, it is easy to overreach at the federal level. Again, 
my point is if the research is done, if it is relevant, if we have good 
ways of transmitting, and if educators are held responsible for the 
results they will use it. You don’t have to force them to do it. 

Chairman Hunter. Answer this, too: How do you make sure, 
then, if you have the data, and you have the best practices, and 
you have the research that shows what should be taught, how do 
you — if you don’t want to get down in the weeds on implementa- 
tion, because you don’t want to because every — there is no silver 
bullet for — you could have two schools side by side on two different 
blocks and they would require different implementation. How do 
you guarantee, then, that the research is taken to bear in that 
school, or do you? Do you just let — kind of let the — if they want to 
use it then they can use; if they don’t want to use it then they don’t 
have to use it? 

Mr. Whitehurst. Well, you certainly raise a very great chal- 
lenge. But there is research relevant to that challenge, and it is re- 
search on implementation. So we are developing a knowledge base 
about the ways you need to transmit knowledge, the ways you need 
to provide professional development around that knowledge, the 
way you need to monitor implementation to make sure that a pro- 
gram is being carried out well. 

So I think, you know, on the forefront of education programs that 
are being shown to be effective is a very strong component having 
to do with implementation. So you are not asking school personnel, 
you know, to take something off the shelf and figure out how to im- 
plement it. The implementation is built into the program — to the 
program itself. 

Chairman Hunter. Thank you very much. 

And as my time is expired, I would like to recognize Mr. Holt for 
5 minutes. 

Mr. Holt. Thank you. 

Actually, we have a very broad topic today, or a collection of 
many topics. Of course, we must not forget that what underlies all 
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this is that research communicated well and made relevant is our 
best protection here, and also the teachers’ best protection, against 
allowing one’s deeply held beliefs and ideologies from blinding 
them, us, to reality and best practice. — And we need to make that 
research practical. 

We are talking today about national research. We are talking 
about the National Center for Educational Statistics. We are talk- 
ing about comprehensive centers. We are talking about the regional 
labs that are — I like to think of as akin to the Agricultural Exten- 
sion Service that maybe your father the farmer actually used, be- 
cause there are best practices that come from the federal level that 
a farmer would depart from only at his own peril. 

But we are also talking today about local data. I mentioned in 
my opening remarks that I will be reintroducing the Metrics Act 
to provide federal assistance to local agencies to apply data and use 
it locally. Let me start with Mr. Fleischman and then Mr. Smith 
to ask, what do you think is a useful federal role in supporting 
local data system development, and can you give me examples of 
how that has or how it could be used well? And then as time allows 
we will go to other questions and other witnesses. 

Mr. Fleischman. Yes. Thank you for that question. 

And first of all, I would say there are a couple of very good recent 
reports out of the Data Quality Campaign that I think are worth 
looking at. One came out last month and one just came out this 
month, and it looks at the connection between state data systems 
and how districts use that. I think they have a number of rec- 
ommendations in there that are valuable to keep in mind, because 
in the end, the state data systems have to be used at the district 
level, the school level, and the classroom level. So the question is 
how to better connect all of those pieces of the system. 

Going back to this notion of focus on the consumer or the user 
of the data, I think it is really important — and the Data Quality 
Campaign cite some examples of how states have worked really 
well with their end users — to create the support mechanisms nec- 
essary so that data is not used for compliance purposes but for con- 
tinuous improvement purposes. I will cite one specific example at 
the local level — for me a local level is both the state and the dis- 
trict — through our Regional Educational Laboratory work right 
now. We are using some of the framework that is provided by a 
number of practice guides. These are kind of taking best available 
evidence and then helping educators by providing them practical 
recommendations. 

We are working with several of those practice guides, including 
one on turnaround and one on data-driven decision making, with 
a set of local schools and local school districts in the Columbia 
Gorge area of Oregon to help them use in a rapid increase cycle 
where they look at their data continuously for the purpose of im- 
provement. So they take the action, they look at the data, they 
focus, and they 

Mr. Holt. To give Mr. Smith some time to answer that 

Mr. Fleischman. Sure. 

Mr. Holt [continuing]. Let me turn to him. 
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Mr. Smith. Thank you very much. I think, you know, the driv- 
er — why do people in the field want to do anything? Why do they 
want to look at data? What data do they need, and so forth? 

And it has been my experience as commissioner and super- 
intendent that the school leadership and district leadership is driv- 
en by data because of the issues around accountability and trying 
to find out if they are being effective or not, how do they bench- 
mark their work over the course of the year, if they need to make 
corrective action. And so I think there is a — based on the structure 
we have in our nation, there is inherent desire on the part of dis- 
tricts and schools 

Mr. Holt. My specific question is, can we help local educational 
agencies use data better, and do you have examples of that? 

Mr. Smith. I think — I don’t know if that is — the federal govern- 
ment needs to be involved with that or that is more of a state and 
district issue. I think that in terms of 

Mr. Holt. But it is not happening. 

Mr. Smith. I would share that there is a great deal of data anal- 
ysis going on in schools every day trying to determine the effective- 
ness of instruction that takes place. Connecting that between what 
is effective — what do I do when I find that the work that is going 
on in the classroom or schools isn’t working isn’t effective? What 
is the solution? Where do I go? 

That is where the breakdown is. It is not so much that I don’t 
know that School X, as a commissioner, is failing and has failed. 
What do I do? What solution set do I bring to it and apply to it? 

Mr. Holt. Thank you. 

Chairman Hunter. Thank you, Mr. Holt. 

I would now like to recognize Mrs. Roby, from the great state of 
Alabama. 

Mrs. Roby. Thank you, Mr. Chairman. And I, too, know how 
quickly 5 minutes goes by, so we will just jump right in. 

Dr. Whitehurst, you talked with the chairman a little bit about 
implementation. I just want to expand on the fact that in your tes- 
timony you said that Congress should focus on creating incentives 
for practitioners to want to incorporate findings from the best re- 
search into their programs centered around the performance of 
schools. 

And we hear that word a lot in here — incentives — and rarely are 
we given the opportunity to hear specifics as to what those incen- 
tives might be and how the federal government actually offers 
those incentives. So could you expand on that a little bit? 

Mr. Whitehurst. Yes, I can. I think there are two categories of 
incentives. The one is top-down regulatory incentives, where, as 
has been the case in No Child Left Behind, states have to define 
targets for performance of schools. Schools that are not meeting 
those targets face various consequences. You have similar sort of 
mechanisms structured around positive incentives in Race to the 
Top. 

But somebody at either the state or the federal level is saying, 
“This is what you need to do, and here are the things that are 
going to happen if you don’t do those things well.” There is decent 
evidence that that kind of top-down accountability has effects, and 
you will hear practitioners like Dr. Smith say, “Well, we are con- 
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cerned with accountability. We wanted to do something about the 
schools that were failing, as defined by the accountability system.” 

The other form of accountability is market-based accountability. 
Your school is failing not because you are not reaching some target 
set by the state; your school is failing because parents don’t want 
to send their kids there and they have other places to send their 
kids 

Mrs. Roby. And I guess that is the — and sorry for inter- 
rupting — 

Mr. Whitehurst. Yes. 

Mrs. Roby [continuing]. That is the problem, because not in 
every school district do you have that opportunity to make that 
choice. And that is the real rub is that if my school is failing and 
I don’t have a choice to go anywhere but that school then that in- 
centive doesn’t exist. 

Mr. Whitehurst. Well, I think that it is possible for the federal 
government to do more to incentivize states to incentivize districts 
to allow at least public school choice. Now, if you live in a commu- 
nity in which there is only one elementary school obviously you are 
not going to have choice. But if you live in a community like Wash- 
ington, D.C., in which there are hundreds of elementary schools, if 
you can choose among them based on good information on how you 
are performing, that sends a very strong accountability message. 

And again, I think that is a different form of accountability. It 
is not fundamentally incompatible with top-down accountability, 
but I believe we need more of it. And the best evidence is that 
when that accountability is in place the schools that are subject to 
the loss of students improve, and that parents — low-income par- 
ents — given good information, will choose a better school than the 
school that the district assigns their child to, and their kids will do 
better as a result. 

So incorporating that kind of market-based approach in the ac- 
countability system, I think, is a way to go, and we could do more 
of that at the federal level. 

Mrs. Roby. And certainly we know that part of that challenge, 
too, is how to get that information into the hands of the parent, 
and that is a whole ’nother topic of conversation. But thank you for 
your answer. 

Dr. Hoxby, how can IES effectively partner with the private sec- 
tor to conduct quality research and make it accessible to teachers 
in the classroom? 

Ms. Hoxby. Well, I think in many ways the best way to answer 
that question is to explain what happens abroad in other countries, 
because there is no partnership between the private sector and the 
government in most other countries, and as a result, their edu- 
cation research is very narrow. The government really has a mo- 
nopoly on what are the interesting questions and what are the 
right ways to answer those questions. And also, they don’t tend to 
have very much advancement in terms of their scientific methods 
for answering those questions. 

In the U.S. we do have a pretty effective partnership already be- 
tween the government and philanthropic organizations and univer- 
sities. And I think if we look at what something like the Gates 
Foundation does, it starts interesting, innovative programs, some of 



32 


which are never brought to scale; it has those programs evaluation, 
sometimes by university researchers, sometimes by other private 
sector researchers, like Mathematica, a contracting organization, 
and then it makes decisions about which of these programs to con- 
tinue and which of these programs should be discontinued. 

I think that is a fantastic role for philanthropic organizations be- 
cause it is their money and they want the, if they want the credit 
for being innovative they should take the risks of being innovative. 

I think universities play a much bigger role in the United States, 
as well. I don’t know whether we consider that the private or the 
public sector. I suppose it depends on the university. 

But I think the key thing that the universities do is that they 
will do basic research, and basic research is important not be- 
cause — basic research is research that doesn’t apply immediately to 
policy, but it usually applies to policy just one step down the road. 
So as an example, all of the research that has come out recently 
on teachers — the effects that teachers have on students — some 
teachers have much more positive effects, some teachers have 
much worse effects on students. 

That is all basic research because all it tells us is that we know 
that teachers differ a great deal. It doesn’t tell us how we ought 
to pay teachers, but we need to know that if we are going to then 
consider teacher pay policies. 

Mrs. Roby. Thank you so much. My time is expired. 

Chairman Hunter. Thank you. 

Mr. Scott is recognized for 5 minutes. 

Mr. Scott. Thank you, Mr. Chairman. 

And I want to welcome Dr. Smith. You just breezed by, Mr. 
Chairman — breezed by the highlight of his career, and that is su- 
perintendent of the Newport News, Virginia public schools. 

Dr. Smith, it is good to see you again. When you were super- 
intendent I think that they had the research — federal research was 
under — I forget what it stands for, but it is OERI. Did you ever use 
any of that research in Newport News? 

Mr. Smith. No. 

Mr. Scott. In your other capacities have you used federal re- 
search in your — you asked Dr. Whitehurst for research on things 
you needed. Was the research there? 

Mr. Smith. No. We have. We got it and we have used it a lot, 
and — from a variety of sources, but research from the federal gov- 
ernment, where available and applicable, I would — we have a — in 
Florida we have a very well developed reading office and we con- 
stantly stay up with the most current research on reading, and so 
forth. 

And I would say that, you know, in the field — and again, it var- 
ies a bit from state to district to school; perhaps it goes back to the 
question asked earlier about data. A lot of research can inform the 
work in general and overall. Day to day, a lot of schools — most 
schools I run into — do have good data, or they have data; they don’t 
have good data, and they actually are doing research on their own. 

Mr. Scott. But that data and research are two different things. 
If you have done some research and found out what works, do you 
report back to whoever did the research to see if it worked in your 
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locality? Because I suspect that some very successful programs 
would work in one setting and not in another. 

Mr. Smith. We do find that the application of what is learned 
can vary from setting to setting. And again, you know, sometimes 
information we will gain helps us with that; what practices work 
best with students that have limited experiences when they come 
to the classroom, or the converse. 

Adaptive testing is helping us now, because we have some work 
with adaptive testing that allows us to measure up and measure 
down, and so we can be more student-centric in our review and try- 
ing to find the right kind of solutions to, you know, based on the 
research, on what needs to be done. 

Mr. Scott. Thank you. 

Dr. Hoxby, you indicated the importance of making sure you get 
the best proposals. Does the Institute of Educational Research wait 
for proposals or do they put out RFPs of subjects that need to be 
studied? 

Ms. Hoxby. Both, and I think both are important. By putting out 
priorities IES does get researchers engaged in questions that are 
important for policy, especially federal policy makers, and certainly 
some of the priorities come right out of federal programs that are 
funded. I think those are very important priorities. 

But I don’t think we want IES to be establishing all of the prior- 
ities simply because sometimes the most important innovative pro- 
grams really come out of nowhere, or out of some educator’s idea, 
out of a particular school, out of a particular school district that is 
doing something innovative, and then often those proposals flow 
into IES. So I think we have a pretty good balance at this point 
of establishing priorities and attracting researchers to them, but 
also allowing researchers to notice what is going on out there in 
the field and then bringing that into IES and saying, “I can evalu- 
ate this.” 

Mr. Scott. And with that process do we have information — I 
mean, if we want to reduce the achievement gap and the school 
board gets together and says, that is our priority; we want to — is 
there somewhere you can go to get research on what they need to 
do? 

Ms. Hoxby. Well, I think that is a tricky question. Ideally they 
could go to the What Works Clearinghouse, which is part of IES, 
and look up something like, “How do I close the achievement gap,” 
but it is really not that straightforward. What the What Works 
Clearinghouse would tell you is maybe what reading curriculum 
works best, or what math curriculum works best, or it might give 
you a good sense of whether charter schools are doing better than 
public schools in a particular domain. 

So we still have a problem in that the school superintendent 
really has to put all of these pieces together, and I do think that 
is the gap everyone is identifying. 

Mr. Scott. So we have 15,000 superintendents home making 
their own process and no central research to help them out. Is that 
what we have, Dr. Whitehurst? 

Mr. Whitehurst. Not exactly. We certainly have research to 
help them out. I agree with Dr. Hoxby that often a practitioner will 
come at the problem with different slices than the research commu- 
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nity has, and so there is a challenge in putting it together and an- 
swering the practitioner’s immediate problem. 

Part of this is simply a lack of knowledge. We have not been at 
this game seriously for very long, and one of the frustrations I had 
when I was the director of IES is people would ask me what to do 
and they would want an answer, and I could not give them an an- 
swer based on the knowledge base that we had created. 

So some of it is a problem of translating what we know more ef- 
fectively. Some of it is a problem of our just not knowing yet the 
best way to go about doing what needs to be done. 

Chairman Hunter. Thank you. 

Mrs. Foxx is recognized for 5 minutes. 

Mrs. Foxx. Thank you, Mr. Chairman. 

And I want to say that this has been a very enlightening panel. 
I want to thank all of you for coming today. I have had a little ex- 
perience in this area, and am very fascinated by the subject of re- 
search. 

Serendipitously, over the weekend I read an article from Na- 
tional Review Online — and I am sorry Mr. Scott left — it is called 
“Closing the Achievement Gap.” I don’t know the people who wrote 
it; Reihan Salam and Tino Sanandaji are their names. But it is a 
fascinating article that brings up the issue of research and how dif- 
ferent people can look at the same research in different ways. And 
I think that is an underlying issue that is pretty important. 

I want to make a couple of comments and then ask some ques- 
tions. As I said, I have been in this field for a long time, and as 
you all were talking and as I have read your statements, I kept 
coming back to that statement, “Everything I need to know I 
learned in kindergarten.” 

Dr. Whitehurst, while you said we are in this field only a short 
time, the comment you made about what we have learned from re- 
search, teachers vary dramatically in effectiveness. A very effective 
teacher compared to a very ineffective teacher can create achieve- 
ment gains for a child in 1 year that can wipe out a third of the — 
haven’t we always known that? I mean, did we need to do research 
to figure out that there are some good teachers and some not-so- 
good teachers? I mean, why did we have to have research to teach 
us that? 

And I guess the question that I would like to ask and quickly get 
a quick answer, if I could, from all of you — very quick answer: Is 
there research that has not been done that needs to be done? Just 
give me two or three words, each one of you, if you would. What 
don’t we know that we should know? 

Dr. Whitehurst? 

Mr. Whitehurst. Well, there is a lot we don’t know about effec- 
tive curriculum, particularly how to deliver it digitally. We are 
moving into a digital age, and being able to use that medium would 
be extraordinarily important. 

Mrs. Foxx. Okay. 

Dr. Hoxby? 

Ms. Hoxby. I think the most obvious gap is that we don’t know 
how well teacher incentives work for improving teaching in the 
classroom. Most of our studies are now from other countries, not 
from this country. 
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Mrs. Foxx. Mr. Fleischman? 

Mr. Fleischman. I think we need more research on data use, 
how to use it effectively, and also, across the board, implementa- 
tion — how to implement more effectively. 

Mrs. Foxx. Mr. Smith? 

Mr. Smith. Yes. I would say how to help classrooms to better 
adapt to the variability that comes to the teacher every day — the 
high flyer, the high performer — and still be able to adapt to the 
need of the child that is struggling on a given topic. 

Mrs. Foxx. I have one child. She is an average kid, and I always 
felt sorry for every teacher she had because she was always in a 
class — we were in a small community — where they had very, very 
bright kids and kids with major challenges, and a whole lot of kids 
right in the middle. And I felt sorry every year for those teachers 
because they had that range to deal with, and I think you have 
identified a very important point. 

The other thing I would like — Dr. Hoxby, you brought this up so 
let me direct the question first to you, and then if others want to 
respond I would be happy for you to do that. You mentioned the 
Gates Foundation and what they have been doing. Has the Gates 
Foundation been more effective in its implementation of what they 
have learned than the federal government has been, or other places 
like the Gates Foundation? 

Ms. Hoxby. I wouldn’t say that they have been more effective. 
I would say they have looked more at speculative programs as op- 
posed to established programs. I think that is a difference between 
the role of the philanthropic organization and the government. 

I would say they are also faster at shutting down unsuccessful 
programs. That is the other thing: When they figure out that some- 
thing is unsuccessful it doesn’t take them a couple of years to shut 
down; it takes them a couple of months. 

Mrs. Foxx. Thank you very much, Mr. Chairman. 

Chairman Hunter. Thank you. 

I would like to recognize Mrs. McCarthy for 5 minutes. 

Mrs. McCarthy. Thank you. Actually, I am finding this quite 
fascinating, and I have got 2 million questions in my head as we 
go through all this. 

One of the things that I have always been kind of looking at — 
you know, we have great people that want to be teachers. Yet we 
find when they get into a school to be teachers most of them are 
put into lower grades. I am just wondering if the research has been 
out there on what our teaching colleges are doing to make sure 
that teachers are well prepared to go into the lower grades. Be- 
cause what I have found in talking to an awful lot of young teach- 
ers when they first start, they felt totally unprepared to be teach- 
ers. A lot of them have left within 5 years because they felt that 
they had the biggest responsibility to take the youngest and to give 
them the best, and yet they felt they were not capable of doing 
that. 

I would just be interested because I think when you talk about 
the digital age that is coming in — our younger people that are 
graduating, hopefully they are going to be more focused, because 
that is the way kids want to learn today. I think that is one of the 
problems that we are seeing in our schools, also. 
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I guess the other question that I would have would be that when 
the data come in and if you have someone that is a superintendent 
or a principal that is not interested in data or doesn’t even have 
time to look at data, is the state prepared to be able to get that 
information down when they see those schools are failing? I will 
throw that open to everybody. 

Oh, and before I forget, I have an article from the RAND Edu- 
cation on some research that I would like to submit to the com- 
mittee, because I am a supporter of charter schools. I also believe 
it is not the silver bullet that everybody is looking for. With that, 
Mr. Chairman 

[The policy brief, “Increasing Participation in No Child Left Be- 
hind School Choice,” may be accessed at the following Internet ad- 
dress:] 


http .7 lwww.rand.orglpubslresearch_hriefslRB9424tindexl.html 


Chairman Hunter. Without objection. 

Mrs. McCarthy. Thank you. 

Mr. Whitehurst. I will go first, since I am on the right here. 
There are good survey data indicating that teachers in general 
have the reactions that you have just described. They feel badly 
underprepared for the jobs that they have to do. It does result in 
a lot of loss from the profession. 

Innovations around that are several. Some districts are setting 
up their own teacher preparation programs so that the practical ex- 
perience is directly related to what the district wants to provide. 
We have programs like Teach for America that are providing alter- 
native pathways in teaching that bypass the traditional school of 
education preparation routes. 

But clearly, we need to do a better job in preparing teachers for 
the jobs that they have to do, that we are, for the first time, to Mrs. 
Foxx’s issue, actually able to measure teacher effectiveness rather 
than just having the intuition that there are good and bad teach- 
ers. It allows us to tie the performance of classroom teachers to 
their preparation institutions, so for the first time the colleges of 
education can be held responsible for the quality of instruction that 
the teachers provide. So I will handle that question, and I will let 
my colleagues take on that or other ones, if they wish. 

Ms. Hoxby. Let me just follow up on that, and I won’t repeat 
what Dr. Whitehurst said. But we do know that if you look now 
at data it does not appear to be the case that teachers who are edu- 
cated in different channels are systematically better or worse than 
one another. Teachers who are educated through alternative teach- 
er channels often look about the same, in terms of their perform- 
ance, as teachers who go all the way through a traditional ed 
school and it takes them 6 years to get their degree. 

And that is somewhat disturbing because it means that whatever 
it is that we are doing in the training, it does not systematically 
work. I think these days we have to look backwards, the other way. 
Because we can identify teachers who are effective, we can look at 
the schools that are producing effective — the education schools that 
are producing effective teachers systematically. 
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Another thing that we have learned is that effective teachers are 
good at spreading effective teaching around them. If you drop one 
effective teacher into an elementary school it turns out that the 
teachers who interact with her will also become more effective. 

So we are getting a better understanding of how teachers can 
learn, but it appears, to a large extent, that they learn from one 
another and that they learn from classroom practice, not so much 
just from getting a credential. 

Mr. Fleischman. I think we still have a ways to go in terms of 
what was just said, in terms of having teacher preparation institu- 
tions and other vehicles to prepare teachers to be ready to do the 
job. In part, having been a teacher, there is a lot of on-the-job 
training, and mentoring, and support you need once you get there, 
but there is no question that there could be better preparation. 

In fact, I mentioned before the Data Quality Campaign report, 
just out this month, and they looked at 10 state actions to support 
effective data use. Only one of them was implementing policies to 
ensure educators know how to use data appropriately once they 
have that in place. So there is a lot of work that needs to be done 
in the system to get to the issue that you just raised. 

Mr. Smith. I would just add that, very quickly, one, on teacher 
quality issues, a lot of states aren’t out pursuing the link between 
student achievement and the institution that prepared the teacher 
to enter the profession. There is some work done by some organiza- 
tions to gather more national information on teacher preparation 
and they are having an extraordinarily difficult time getting state 
institutions to give that data up, so having to actually go to a Free- 
dom of Information Request to get that information, so it is a very 
slow process, but very important one. 

On the data side, there — you know, data management systems — 
again, a lot of schools have data; it is not the data that they need 
to focus their attention on the things that are important, and an 
area that I think we need to continue to do research on is what 
are the most effective data management systems out there and the 
most informative for school administrators and superintendents to 
use to, you know, to help drive improvement. 

Mrs. McCarthy. Thank you. 

My time is up. 

Chairman Hunter. Thank you. 

Mr. Hanna is recognized for 5 minutes. 

Mr. Hanna. I would like to use the balance of my time and give 
it to Mrs. Foxx. Thank you, Chairman. 

Mrs. Foxx. I want to thank the gentleman from New York for 
yielding me his time. 

I could not let this panel get away without mentioning something 
that is a particular bone of contention with me, and so far three 
of you have sort of violated my norm on this. You used the term 
“training” in association with human beings. It is a shame to admit 
this, but I remember one thing from my doctoral program, and one 
of my professors said, “You train animals and you educate people,” 
and that has really stuck with me. 

And so especially when I am in education settings I try to point 
that out to people because I want you to think about the fact that 
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we are educating people. Dr. Whitehurst, it is in your material that 
you put out. 

And, Dr. Hoxby, you and Mr. Fleischman both just said it. 

So I would like you to think about whether you want to use that 
term in conjunction with human beings, because I think that has 
something to do with our mindset in education. I really believe lan- 
guage is important, and I am sure you all would probably agree 
with that. 

There are a couple of things that came up. Dr. Whitehurst, in a 
time — we always have limited resources, and I know, as you say, 
in research this has been an area where we have used a very small 
amount of resources, and in some ways have come to it very lately, 
so I agree with you on that aspect of it. 

I would like to start with you again, particularly. Again, I asked 
this question a slightly different way; where could we best use our 
dollars? And something none of you have mentioned, which I would 
like you to think about as you answer that question, we are always 
focused on the teacher, and obviously that is important. The teach- 
er is the person interacting most with the student. 

But I have felt in all my life of being involved in education is we 
never look enough at the structure of education. I believe that most 
of what we do in education is designed for the adults and not the 
children. 

For example, we have known for a long time that adolescents do 
a very poor job early in the morning, and yet, high schools begin 
at 8 a.m. We have ignored that research for the convenience of the 
adults. 

So would you make any recommendations in terms of research 
on structure of education, and would you make some comments 
about that; and again, very quickly so each person has a chance to 
make some comments about that? 

Mr. Whitehurst. Well, if you mean by structure the arrange- 
ment of the school day and the circumstances in which instruction 
is delivered, yes, I think we need policymakers and practitioners to 
pay attention first to the research that we already have. We know, 
for example, that investments in pre-K programs pay a large divi- 
dend, and yet they are typically underfunded. 

We have very strong research demonstrating that high school 
kids’ learning is negatively affected by starting them before they 
are awake in the morning. We have a variety of research that rates 
the organization of the school day. 

And so, you know, I am in favor of — certainly we ought to use 
what we know when we can do that. 

Ms. Hoxby. I think that you are making a very important point. 
I often say to people that the problem in some areas of education 
is not that we don’t know the answer but simply that the stake- 
holders will not listen to the answer or will not use the answer. 

An example of that, for instance, is the longer school year and 
the longer school day. These are just not popular with stake- 
holders, but it — the evidence suggests that they are very good for 
students. So that is a perfect example of where the structure gets 
in the way of improvement. 

Mrs. Foxx. Mr. Fleischman? 
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Mr. Fleischman. Yes. Mrs. Foxx, first of all, thank you for that 
reminder about training. It was made to me by my colleagues last 
week. I didn’t remember. Thank you. 

The one place, I think, where we need to do a lot more re- 
search — it is also on structure — is the connection between sec- 
ondary school and college-going — college readiness, college attend- 
ance, college success. That is one of those places where there are 
two different structures coming together and we need to better un- 
derstand how the secondary school can do a better job and how the 
institutions of higher education can do a better job to ensure the 
students’ success. 

Mr. Smith. I would just agree with you that a longstanding belief 
of mine that schools don’t fail; districts fail — that when you find — 
that districts are the ones responsible for setting the policies that 
drive much of what goes on in schools. And to fix schools school- 
by-school is extraordinarily challenging because the envelope that 
it operates within is usually broken also. 

Mrs. Foxx. Again, Mr. Chairman, I want to thank you very much 
for your tolerance, and I want to thank the panel. 

You have been a 

Chairman Hunter. I thank the gentlelady. 

I would now like to recognize my neighbor in San Diego, Mrs. 
Davis, for 5 minutes. 

Mrs. Davis. Thank you, Mr. Chairman. 

Thank you to all of you for being here. You have all had a great 
deal of experience in this area, and I am wondering what you see 
has been the best way that research has been disseminated to 
teachers. 

I think, Dr. Hoxby, you mentioned that we do know — and I have 
heard Bill Gates say — that putting new teachers or teachers who, 
perhaps, aren’t doing as well in front of teachers who are doing 
very well and seeing how they engage with students, and how they 
get so much from students is beneficial. We don’t see that enough. 

It seems to me we fail to do that. We fail to provide the resources 
so that we can have those really great teachers in front of new and 
unsuccessful teachers. 

Is that one way that we could do that? Have you seen that? How 
do we do that? 

Because going to a practicum 1 day is not going to do it. We 
know that. How do you think it works? 

Ms. Hoxby. Well, let me first say that one of the things we know 
because of IES is that a lot of the professional development pro- 
grams that are highly regarded in the United States don’t appear 
to have the effects that we — that they are intended to have. So sim- 
ply putting teachers into a professional development program does 
not necessarily have big effects. 

I think that we — one thing that we lack in the United States 
that other countries have, and particularly England has, is a sys- 
tem of school inspectors, and these inspectors it sounds like some- 
one is just coming down to inspect your school like inspect your 
house, but that is not really what they do. They come into your 
classroom, they observe you for several days. They are experts; 
they have all of the data on what is happening in your classroom 
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and the achievement of your students; they have the diagnostic 
data. 

And they sit down with the teacher at the end of the inspection 
and try to relay best practices to the person. And they have an in- 
tense experience that we simply do not have paralleled in the 
United States where a principal would often spend as little as 
maybe an hour in a teacher’s classroom each year. 

Mrs. Davis. Right. Yes. 

Mr. Smith. I would just share a couple thoughts. One is that you 
have to get the information out to where people go, where they at- 
tend, whether it be through national conferences, or whatever, but 
there has to be a strong push to disseminate good quality research 
through the normal channels. 

The second, what I have learned from my experience in Florida’s 
commissioner is that there is, I think, a great deal of dissemination 
that could be done — I don’t think it is being done yet in — at least 
consistently across the nation — by working with a combination of 
state departments of education and legislative committee staff in 
state government, where there is a keen interest in taking research 
findings, be it school day, or connecting teacher quality with teach- 
er preparation, and trying to drive that into state policy and state 
statute. 

Mrs. Davis. So where — is that a federal role? Should there be 
some way — we are all familiar with the military and defense re- 
search, and others in environmental and energy areas. 

Mr. Smith. I think the dissemination — and again, I think — you 
know, the dissemination is, you know, if, I would say that if some 
sources of information, be it regional labs or whatever, my friend 
here, but if they had to depend on checks coming in for how much 
service was provided they might go broke within a month. And so 
I think that, again, there needs to be that consumer-driven process. 
This research is critically important to us. 

Mrs. Davis. Mr. Fleischman? 

Mr. Fleischman. What I would add is not to forget the human 
factor. Just in the same way that we are having a dialogue right 
now and we are learning about something, I think that a lot of the 
learning that takes place takes place in context with people doing 
their jobs and then having better data and better research to in- 
form that. 

A good example of that through the Regional Educational Lab- 
oratory system are the so-called bridge events, where we take 
things like the practice guides, which are based on the best avail- 
able research, and give practical recommendations, and then work 
with folks out in the field. We just recently held one on rural school 
turnaround where we were looking at the recommendations of 
rural school turnaround, looking at the school improvement grant 
models, working with rural school educators and state departments 
of education, and working through the process of learning how to 
apply that in real time for real problems. 

Mrs. Davis. If I might, but really — oh, looks like my time is up. 
I can’t do that. 

Thank you, Mr. Chairman. 

Chairman Hunter. I thank the gentlelady. 

Mr. Barletta is recognized for 5 minutes. 
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Mr. Barletta. Thank you, Mr. Chairman. 

Dr. Whitehurst, current law requires that education programs be 
supported by scientifically-based research. Based on your past ex- 
periences in the field of education policy and your current work at 
the Brookings Institution, how do you define scientifically-based re- 
search? 

Mr. Whitehurst. Research in education that draws on the meth- 
ods that are the canon for the social and behavioral sciences is sci- 
entifically-based research. People are trained to do it. People who 
are trained to do it recognize it when they see it and recognize it 
when it is not happening. 

It is a moving game in that the methods improve and our ability 
to focus those methods on questions that are important changes 
over time. And I think there is, you know, a congressional role in 
mandating that federally funded research meets high standards for 
its scientific base. And it is also ultimately the role of the science 
community, the research community, to define specifically what 
that means, because again, it will change and advance over time. 

Mr. Barletta. And upon the reauthorization of ESEA, how do 
you think this definition needs to be revised? 

Mr. Whitehurst. The current definition, I think, is a pretty good 
one. I think you have a choice either to leave it out and leave the 
definition up to the research community or to take what is there 
and fine-tune it where necessary. 

I think it would be a mistake to take the current definition and 
water it down because that is a signal that we will be moving back 
to where we were 15 years ago, where what passed for education 
research was frequently a subject of derision in any department 
and any university except the education school. 

Mr. Barletta. Dr. Hoxby, same question: How do you define sci- 
entifically-based research, and do you think the definition needs to 
be redefined? 

Ms. Hoxby. I really define scientifically-based research in the 
way I would define it in medicine, or physics, or anything else. It 
is the use of the scientific method. 

And one of the ways that we know we are doing scientific re- 
search is that we should be able to come to conclusions that are 
based on the data and the logic as opposed to based on our pre- 
suppositions. Sometimes you should realize that the data overturns 
your presuppositions. That is the scientific method. 

I don’t think that we need to take science out of ESRA reauthor- 
ization. I completely agree with Dr. Whitehurst that the situation 
we were in 15 years ago is so much worse than the situation we 
are in right now that we need to keep that scientific standard in 
the legislation. 

At the same time, it is almost impossible to define what scientific 
method is because it is a moving target, and that is a good thing, 
right? We wouldn’t want it to be true in medicine that the science 
of today was the same as the science of 10 years ago. 

And similarly in education, one of the great points of using the 
scientific method and requiring that it is used is that the methods 
actually improve because we realize we can’t answer this sort of 
question so we need to have a new method to answer that sort of 
question, or this question has been answered very imperfectly so 
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we need to develop a new method. We want to actually keep the 
development of methods so that 20 years from today we are not 
just in a different place in terms of what we know on education but 
we are in a different place in terms of what we can know because 
we have better methods. 

Mr. Barletta. And, Dr. Smith, as a past classroom teacher and 
school principal, how do you define scientifically-based research? 

Mr. Smith. You know, I have worked with this a lot and I don’t 
know — I don’t think that the definition needs to be changed a lot. 
Because, again, I came out of the world when we didn’t really have 
any research. Whatever felt good and seemed right and the adults 
were comfortable with seemed to be okay. 

And so I think we have made huge strides forward. I think the 
question is what drives the application of the definition, and is it 
being driven by — strictly by researchers that don’t understand the 
connection and application in the real world or is it — is working 
within that definition in a way that gives you real-world, timely 
answers? 

And I think, as in medicine, you can deal with an epidemic in 
a lot of ways — you can define it very narrowly, very rigorously, 
with controls, and by the time the epidemic has already taken its 
toll. Or you can find other ways of working on very scientific, high- 
ly respected results that give you more practical — mirroring the 
conditions that exist at the moment in a timely fashion. 

So I think how that gets gauged and who helps guide the struc- 
ture of the research I think is the key. 

Mr. Barletta. Thank you. 

Thank you, Mr. Chairman. 

Chairman Hunter. I thank the gentleman. 

I think that is all the questions we have. 

I would like to thank the witnesses and finish by just saying 
this: I am optimistic because — not necessarily because I think we 
are all smart people and we can all handle this, but because tech- 
nology, and especially adaptive learning technology, you know, it is 
going to be working and it is going to be implemented at some 
point, I would say, over the next decade or two, and — I mean, if 
they already have adaptive video games, things that work that way 
where the smartest kids get to learn as the smartest kids do and 
excel and the average kids get to have the education curriculum 
matched to them, and so forth for every learning possibility. 

So I am optimistic, one, and I do think that sunshine and data 
can create accountability. I think just the fact that if it is easy to 
consume and it is easy for all the players to be able to read it, and 
understand it, and see who is winning and who is not, and where 
they should send their kids to school and where the educators want 
to go to work at, I think that is a big motivator for everybody at 
every level for all the different stakeholders. 

So thank you, again, to our witnesses. 

And there being no further business, this subcommittee stands 
adjourned. 

[Additional submissions of Mr. Holt follow:] 
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Prepared Statement of the Learning and Education 
Academic Research Network 

As the panel considers reauthorization of the Education Sciences Reform Act of 
2002 (ESRA), the Learning and Education Academic Research Network (LEARN Co- 
alition) is pleased to submit this statement in support of this process and, in par- 
ticular, to highlight the role of research-intensive colleges of education in fulfilling 
the potential of this landmark legislation. 

The LEARN Coalition was formed nearly seven years ago to advocate for quality 
education research at the federal level. Our institutions are dedicated to the most 
rigorous standards for designing and executing the critical research needed to in- 
form better teaching, stronger schools, and, most importantly, higher performing 
students. The Department of Education, the National Science Foundation, and the 
National Institutes of Health are our primary agency partners in this endeavor. As 
a result of the investments that have been made in education research, new tools 
have been developed to inform teacher practice and impact student performance. In- 
vestments in research across the education spectrum are required to translate what 
we have learned through basic research on the brain, cognition, and learning into 
effective teacher preparation and practice, standards for learning, assessment, and 
inform curriculum development. Investments in educational research and rigorous 
evaluation systems lead to better educational programs, schools, effective teaching, 
and higher student achievement. It is a direct investment in our nation’s economic 
competitiveness. 

Since LEARN’s launch, we have witnessed significant expansion in the federal re- 
sources invested in education research. Moreover, there has been an ongoing and 
collaborative effort between institutions of higher education and government leaders 
to ensure that taxpayer resources are used to address the most important challenges 
for our schools and students. ESRA, and the Department of Education’s Institute 
for Education Sciences (IES), are critical building blocks in an increasingly robust 
education research system. In particular, IES facilitates the kind of research that 
enables the translation of theory into practice using systematic study of phenomena 
from small scale to large. LEARN member institutions contribute to the mission of 
IES by conducting research and setting the knowledge base in a variety of different 
areas including: Teacher Performance Systems; Assessment Standards; Educational 
Interventions for Special Education Students; STEM Education; and English Lan- 
guage Learners. 

As the Committee moves forward with ESRA reauthorization, we encourage care- 
ful consideration of how IES and its programs can fully utilize peer-reviewed, high 
quality research capabilities, such as those found in the nation’s higher education 
community, to drive student achievement. The benefits of this approach include: 

1. Innovation — higher education faculty are at the center of critical, creative 
thinking about the learning and teaching processes, including with interdisciplinary 
teams that combine insights across biologic, environmental, and social factors; 

2. Evaluation — universities frequently work with state and local education agen- 
cies, as well as other stakeholders, to conduct field-based research and evaluation 
that promotes timely understanding of what works; and 

3. Dissemination — through a variety of education, publication, and engagement 
tactics, higher education participants are a critical link for translating new knowl- 
edge into practice, on both a focused and larger scale. 

The LEARN members are prepared to provide the Committee with a comprehen- 
sive perspective on how research-intensive higher education institutions contribute 
to better student outcomes. The ESRA reauthorization process clearly is an oppor- 
tunity to accelerate and expand the nation’s efforts here through sound evidence de- 
velopment and use. Our institutions are committed to being at the forefront of pro- 
ducing these student performance solutions and to working with federal policy- 
makers to improve student outcomes. 

LEARN MEMBER INSTITUTIONS 

Indiana University, W.W. Wright School of Education 
Iowa State University, College of Human Sciences 

New York University, Steinhardt School of Culture, Education, and Human Devel- 
opment 

Purdue University, College of Education 

Rutgers University, Graduate School of Education 

State University of New York at Buffalo, Graduate School of Education 

Syracuse University, School of Education 

Texas A&M University, College of Education and Human Development 
The Ohio State University, College of Education and Human Ecology 
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University of California — Irvine, Department of Education 

University of California — Santa Barbara, Gevirtz Graduate School of Education 

University of Florida, College of Education 

University of Illinois Urbana-Champaign, College of Education 

University of Iowa, College of Education 

University of Maryland College Park, College of Education 

University of Minnesota Twin Cities, College of Education and Human Development 

University of Pittsburgh, School of Education 

University of Southern California, Rossier School of Education 

University of Virginia, Curry School of Education 

University of Washington, College of Education 

Vanderbilt University, Peabody College of Education and Human Development 


[The report, “From Compliance to Service: Evolving the State 
Role to Support District Data Efforts to Improve Student Achieve- 
ment,” may be accessed at the following Internet address:] 

http://dataqualitycampaign.org/files/From%20Compliance%20to%20Service.pdf 


[Questions submitted for the record and their responses follow:] 


U.S. Congress, 

Washington, DC, December 5, 2011. 


Dr. Caroline Hoxby, 

Department of Economics, Stanford University, 579 Serra Mall, Stanford, CA 94305. 

Dear Dr. Hoxby: Thank you for testifying before the Subcommittee on Early 
Childhood, Elementary and Secondary Education at the hearing entitled, “Education 
Research: Identifying Effective Programs to Support Students and Teachers” on 
Wednesday, November 16, 2011. I appreciate your participation. 

Enclosed are additional questions submitted by members of the Committee after 
the hearing. Please provide written responses no later than December 19, 2011 for 
inclusion in the final hearing record. Responses should be sent to Dan Shorts of the 
Committee staff who can be contacted at (202) 225-6558. 

Thank you again for your important contribution to the work of the Committee. 
Sincerely, 


Duncan D. Hunter, Chairman, 

Subcommittee on Early Childhood, Elementary and Secondary Education. 


REPRESENTATIVE DUNCAN HUNTER (R-CA) 

1. Given that the focus of this hearing is to examine the most effective ways of 
utilizing student research to help teachers better understand students’ instructional 
needs, it would be helpful to hear your thoughts on computer adaptive assessments. 
These assessments adjust automatically to each student’s ability level, generating 
more difficult questions if the student is answering correctly and easier ones if the 
student is answering incorrectly. In doing so, these assessments enable teachers to 
pinpoint the proficiency level of each student across a range of subjects that cor- 
respond with the standards set by a state’s curriculum. 

There are a few states who have already implemented computer adaptive assess- 
ments as a tool of measuring student achievement and growth — including Oregon — 
and a number of others who are interested in following suit, given that computer 
adaptive assessments provide essential and timely data that can more accurately il- 
lustrate student placement, student growth, and instructional needs. 

Can you provide the Committee with your views on computer adaptive assess- 
ments and whether they can be of benefit to teachers, administrators, parents, and 
ultimately students? 

2. How does education research play a role in providing reliable information to 
parents? How can the federal government aid states and school districts in improv- 
ing these efforts? 

3. With such high standards for scientific evaluation, how can the federal govern- 
ment ensure that the research methodology is not overly cumbersome, leading to ar- 
tificial results that are not relevant in a dynamic and fast-changing classroom? 
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REPRESENTATIVE VIRGINIA FOXX (R-NC) 

4. During the hearing you mentioned that other countries (specifically in Europe 
and Latin America) have better administrative data sets than the United States, 
and you could therefore do better research in other countries. I think a specific ex- 
ample you cited was the Dutch school reform and choice movement. Why do other 
countries have better data sets? Is there something in the US prohibiting them from 
collecting the same data sets (i.e. student privacy concerns)? Please expand more on 
why other countries do a better job with administrative data sets. 


Dr. Hoxby’s Response to Questions Submitted for the Record 

CHAIRMAN HUNTER 

1. Given that the focus of this hearing is to examine the most effective ways of uti- 
lizing student research to help teachers better understand students’ instructional 
needs, it would be helpful to hear your thoughts on computer adaptive assessments. 
These assessments adjust automatically to each student’s ability level, generating 
more difficult questions if the student is answering correctly and, easier ones if the 
student is answering incorrectly. 

In doing so, these assessments enable teachers to pinpoint the proficiency level of 
each student across a range of subjects that correspond with the standards set by a 
state’s curriculum. 

There are a few states who have already implemented computer adaptive assess- 
ments as a tool of measuring student achievement and growth — including Oregon — 
and a number of others who are interested in following suit, given that computer 
adaptive assessments provide essential and timely data that can more accurately il- 
lustrate student placement, student growth, and instructional needs. 

Can you provide the Committee with your views on computer adaptive assessments 
and whether they can be of benefit to teachers, administrators, parents, and ulti- 
mately students ? 

Computer adaptive assessments are evaluation tools that are extremely helpful 
because they promote good decision-making at all levels: the classroom level, the 
school level, the district level, the state level, and the federal level. They prevent 
most cheating and crude “teaching to the test.” Because adaptive assessments put 
neither floors nor ceilings on the achievement of students, they allow students who 
are behind or ahead of their grade to be evaluated well. Every student can be appro- 
priately challenged, and no student ever need face a “dumbed down” test. Computer 
adaptive assessments also allow tests from different states to be equated fairly eas- 
ily so that states’ performance can be compared well. 

Let me expand just slightly on some of these points. 

When taking a computer adaptive assessment, a student’s answers to the initial 
questions affect whether he or she offered more or less challenging questions from 
then on. This is the way in which the assessment adapts to the student’s level of 
knowledge and skill. Because students spend their time answering questions that 
efficiently diagnose what they know and do not know, a computer adaptive assess- 
ment delivers a very accurate evaluate of a student’s learning. In contrast, a student 
who is taking a pencil-and-paper test may find that most of the questions are very 
hard or very easy for him or her. For such students, even the best pencil-and-paper 
test delivers only a crude or imprecise evaluation. The results of computer adaptive 
assessments are available immediately, not months after the test is taken. This al- 
lows teachers to use the test results to modify their teaching immediately, in order 
to provide extra instruction in the areas in which the student was weak. Also com- 
puter adaptive assessments provide not only an overall score, which can be used for 
little else but overall evaluation. They provide diagnostic information on exactly 
what knowledge and and skills the student lacks. For instance, a teacher might 
learn that a student can add, subtract, and multiply fractions but does not know 
how to divide one fraction into another. Many assessments give teachers lesson plan 
suggestions as well as results. Thus, the teacher might receive suggested lessons, 
examples, and practice problems for helping students learn how to divide fractions. 

In short, computer adaptive assessments have at least five properties that make 
them very useful to policy makers at all levels: (i) they can be much more accurate 
than a pencil-and-paper test that occupies the same time, (ii) their results are avail- 
able immediately; (iii) their results are useful for diagnosis, not merely for reward- 
ing someone who does well overall or punishing someone who does poorly overall; 
(iv) they generate lesson plans to improve a student’s learning, quickly. 

Computer adaptive assessments prevent outright cheating as well as “teaching to 
the test.” It is easy to make a computer adaptive assessment far more secure than 
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pencil-and-paper tests are at present. The main way in which people cheat on pen- 
cil-and-paper test is inputting or changing answers during the period before the 
legal testing time or in the period after the legal testing time. This method of cheat- 
ing requires no sophistication or cleverness, which is probably why it is the only 
common method. While such behavior is easily curbed by having proctors deliver the 
tests, remain during testing, and remove the tests, states have so far refused to use 
proctors, citing cost concerns. (Whether such cost concerns are legitimate is not at 
all clear, but the point remains that pencil-and-paper tests are not proctored and 
therefore not secure.) In contrast, computer adaptive assessments can easily be de- 
signed to be electronically available only during the legal testing period. While a 
very sophisticated hacker might possibly hack into a computerized assessment and 
enable people to cheat, we have little or evidence that school staff are willing to try 
complex or difficult methods of cheating. 

Because computer adaptive assessments draw upon a very large bank of questions 
and no two students can be expected to take exactly the same test, these assess- 
ments strongly deter “teaching to the test” in its crude form where teachers literally 
train students to answer particular questions. Of course, computer adaptive assess- 
ments do not and should not prevent teachers from helping their students excel by 
having the learn the knowledge and skills likely to be tested by the assessment. 

2. How does education research play a role in providing reliable information to 
parents ? How can the federal government aid states and school districts in improving 
these efforts ? 

Education research can be a much more reliable source of information to parents 
than are schools themselves. This is mainly because researchers do not feel a strong 
need to defend existing policies or support proposed policies. They can afford to be 
objective. In addition, researchers often bring modern scientific methods to bear, 
and these methods are sometimes less familiar to school and district staff. However, 
in order to help parents, it is essential that research (i) be held to a high scientific 
standard, (ii) be as timely as possible, (iii) be made available to parents in an easily 
interpretable form. The federal government can be helpful on all these dimensions. 
By setting high scientific standards for its grantees and contractors, the Institute 
for Education Sciences can strongly encourage the use of the most scientific meth- 
ods. By encouraging schools, districts, and states to build databases that take fairly 
standard forms, the federal government can ensure that research is timely. This is 
because delays in getting data are the main cause of slow research. Most schools, 
districts, and states will build accurate, fairly standardized databases given suffi- 
ciently strong incentives: they are collecting the information anyway. Finally, the 
federal government can encourage federally funded researchers to publish a version 
of their research that is intended for parents and other non-researchers. Non-profit 
organizations often play this “translation” role as well, and it is very important. 

3. With such high standards for scientific evaluation, how can the federal govern- 
ment ensure that the research methodology is not overly cumbersome, leading to arti- 
ficial results that are not relevant in a dynamic and fast-changing classroom ? 

High research standards really have no effect on how quickly we produce re- 
search. It takes no longer to evaluate a rigorously conducted randomized controlled 
trial that it takes to evaluate the same policy in a less scientific manner. In fact, 
many researchers would say that evaluating a randomized controlled trial is faster 
because it is easier. There are three things that do slow education research down, 
and the federal government can improve two of the three. The first thing that 
makes education research rather slow is simply that students change slowly. Even 
the best curriculum in the world does not immediately raise students’ learning. De- 
pending on the intervention, we may have to follow students for a year or several 
years, and there is nothing that we can do about the pace at which students change. 
The second thing that slows down education research is data collection. While eval- 
uation itself is quite fast, data collection is slow. Researchers still obtain data 
through painfully slow processes, in which it is quite normal for researchers to 
spend months if not years soliciting (even begging) for data, making their way 
through layers of administrators, and getting approved in long-drawn-out processes. 
This process need not be slow at all. If schools, districts, and states keep their data 
in a standardized form, in central repositories, researchers would not be forced to 
go through this process. Researchers with strong track records could be given a 
blanket approval so that their data requests were fast-tracked. The third thing that 
slows down education is the reluctance of many educators to provide data or allow 
randomized trials on the policies in which they believe most strongly. Their reluc- 
tance is based on the fear that the research will not validate their strong prior be- 
liefs. Although this problem is not wholly solvable, any intervention that receives 
federal funding could be required to provide data to researchers. This would not 
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only help to ensure that federally funded projects get evaluated well and quickly, 
it would also create a “culture” of evaluation that is still absent in education. 

REPRESENTATIVE VIRGINIA FOXX (R-NC) 

4. During the hearing you mentioned that other countries (specifically in Europe 
and Latin America) have better administrative data sets than the United States, and 
you could therefore do better research in other countries. I think a specific example 
you cited was the Dutch school reform and choice movement. Why do other countries 
have better data sets? 

Is there something in the US prohibiting them from collecting the same data sets 
(i.e. student privacy concerns)? Please expand more on why other countries do a bet- 
ter job with administrative data sets. 

Most European countries and several Central and South American countries have 
much better administrative data sets than the United States. This is largely be- 
cause these countries have more centralized systems of education, and the central 
education ministry requires schools and districts to upload their data in a standard- 
ized format. In the U.S., in contrast, each district has enormous control over its own 
data and reports only a tiny share to its state government: the data elements re- 
quired under its state’s accountability program and under No Child Left Behind. 
While American data bases are improving as states develop longitudinal databases, 
many states have dragged their feet or succumbed to political pressure so that they 
are still far from having good data bases, let alone the comprehensive data bases 
of the aforementioned countries. The resistance to data bases comes from interest 
groups who are afraid that information will expose their lack of contribution to stu- 
dent learning. 

The evidence suggests that the independence of U.S. school districts is a good 
thing for their productivity and their management. If they were centrally managed 
and did not have to compete at all with one another, American school districts 
would likely produce substantially less learning than they do now. However, it does 
not promote efficiency to give each district the right to keep its data in its own way, 
measure things according to its own lights, and create its own idiosyncratic data ac- 
cess procedures. Such lack of standardization greatly inhibits competition and pro- 
ductivity because it makes comparing schools and evaluating policies very difficult. 
We have an analogous situation for firms. Although having firms that are independ- 
ently managed improves competition and productivity, giving each firm the right to 
report data in a completely idiosyncratic way would not make the market better. 
It is important for investors that measures of income, for instance, are fairly stand- 
ardized across firms. Since schools actually engage in a far less diverse range of ac- 
tivities than firms, there is no reason — except for fear of exposure — why they should 
resist standardized reporting much more than firms do. 


U.S. Congress, 

Washington, DC, December 5, 2011. 


Dr. Eric Smith, 

20 Eastern Avenue, Annapolis, MD 21403. 

Dear Dr. Smith: Thank you for testifying before the Subcommittee on Early 
Childhood, Elementary and Secondary Education at the hearing entitled, “Education 
Research: Identifying Effective Programs to Support Students and Teachers” on 
Wednesday, November 16, 2011. I appreciate your participation. 

Enclosed are additional questions submitted by members of the Committee after 
the hearing. Please provide written responses no later than December 19, 2011 for 
inclusion in the final hearing record. Responses should be sent to Dan Shorts of the 
Committee staff who can be contacted at (202) 225-6558. 

Thank you again for your important contribution to the work of the Committee. 
Sincerely, 


Duncan D. Hunter, Chairman, 

Subcommittee on Early Childhood, Elementary and Secondary Education. 


REPRESENTATIVE DUNCAN HUNTER (R-CA) 

1. Given that the focus of this hearing is to examine the most effective ways of 
utilizing student research to help teachers better understand students’ instructional 
needs, it would be helpful to hear your thoughts on computer adaptive assessments. 
These assessments adjust automatically to each student’s ability level, generating 
more difficult questions if the student is answering correctly and easier ones if the 
student is answering incorrectly. In doing so, these assessments enable teachers to 
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pinpoint the proficiency level of each student across a range of subjects that cor- 
respond with the standards set by a state’s curriculum. 

There are a few states who have already implemented computer adaptive assess- 
ments as a tool of measuring student achievement and growth — including Oregon — 
and a number of others who are interested in following suit, given that computer 
adaptive assessments provide essential and timely data that can more accurately il- 
lustrate student placement, student growth, and instructional needs. 

Can you provide the Committee with your views on computer adaptive assess- 
ments and whether they can be of benefit to teachers, administrators, parents, and 
ultimately students? 

2. In your testimony, you talk about the fact that strict application of scientific 
research is often difficult for classroom teachers because of the dynamic nature of 
the classroom. Can you provide some examples of other types of research that are 
beneficial to districts and schools? 


Dr. Smith’s Response to Questions Submitted for the Record 

1. Computer adaptive assessments (CAA) have great potential and we should en- 
courage the thoughtful expansion of its use. CAA can provide the opportunity to 
help teachers more accurately tailor instruction to individual students’ needs for 
both remediation and acceleration. If designed correctly, an adaptive test can also 
be somewhat diagnostic; helping the teacher or a computer program to identify a 
student’s skill deficiencies. Adaptive tests are best used as formative assessments 
that help in guiding instruction and support. The data from adaptive assessments 
should lead to a flexing of the instruction provided a student so that student will 
be able to pass summative standards based exam by the end of the year. 

2. I believe there is a need to research practices that are proving successful in 
the “real world” over time. For example, as a superintendent in Charlotte I learned 
a great deal by sharing strategies and performance data with other superintendents 
that had similar student populations. I learned from them what strategies were 
making a difference in learning outcomes and what strategies were not successful. 
Another example was in Florida where I served as Commissioner. During that time 
we prepared to build a data base that would correlate school performance data and 
teaching strategies. Again, our intent was to learn what conditions led to success 
and what conditions led to failure. This is not to discount more rigorous scientific 
research but I believe we can have a fuller picture by expanding our research strate- 
gies in the “real world”. 


U.S. Congress, 

Washington, DC, December 5, 2011. 


Dr. Grover J. “Russ” Whitehurst, 

775 Massachusetts Ave. NW, Washington, DC 20036-2013. 

Dear Dr. Whitehurst: Thank you for testifying before the Subcommittee on 
Early Childhood, Elementary and Secondary Education at the hearing entitled, 
“Education Research: Identifying Effective Programs to Support Students and 
Teachers” on Wednesday, November 16, 2011. I appreciate your participation. 

Enclosed are additional questions submitted by members of the Committee after 
the hearing. Please provide written responses no later than December 19, 2011 for 
inclusion in the final hearing record. Responses should be sent to Dan Shorts of the 
Committee staff who can be contacted at (202) 225-6558. 

Thank you again for your important contribution to the work of the Committee. 

Sincerely, 


Duncan D. Hunter, Chairman, 

Subcommittee on Early Childhood, Elementary and Secondary Education. 


REPRESENTATIVE DUNCAN HUNTER (R-CA) 

1. Given that the focus of this hearing is to examine the most effective ways of 
utilizing student research to help teachers better understand students’ instructional 
needs, it would be helpful to hear your thoughts on computer adaptive assessments. 
These assessments adjust automatically to each student’s ability level, generating 
more difficult questions if the student is answering correctly and easier ones if the 
student is answering incorrectly. In doing so, these assessments enable teachers to 
pinpoint the proficiency level of each student across a range of subjects that cor- 
respond with the standards set by a state’s curriculum. 
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There are a few states who have already implemented computer adaptive assess- 
ments as a tool of measuring student achievement and growth — including Oregon — 
and a number of others who are interested in following suit, given that computer 
adaptive assessments provide essential and timely data that can more accurately il- 
lustrate student placement, student growth, and instructional needs. 

Can you provide the Committee with your views on computer adaptive assess- 
ments and whether they can be of benefit to teachers, administrators, parents, and 
ultimately students? 

2. How can the purpose and operation of the national research and development 
centers, the RELs, and comprehensive centers be improved upon? Are these entities 
actually serving regional and local needs and assisting states, school districts, 
schools, and teachers to improve student achievement? 

3. The Institution of Education Sciences is responsible for evaluating federal pro- 
grams for their impact on improving student achievement. However, the Office of 
Management and Budget (OMB), the Department’s Office of Planning, Evaluation, 
and Policy, the Government Accountability Office (GAO), and private entities also 
evaluate federal programs for their effectiveness. Is the current system working? 
Are each of these agencies using the same metrics in evaluating programs? Which 
agency is in the best position to evaluate federal programs? 


Dr. Whitehurst’s Response to Questions Submitted for the Record 

CHAIRMAN HUNTER 

Computer adaptive assessment has already been incorporated into psychometri- 
cally advanced assessment programs, including those carried out by the U.S. De- 
partment of Education’s National Center for Education Statistics. For example, the 
Early Childhood Longitudinal Studies, which follow a large sample of children 
through school, carry out all of their student achievement assessments using adapt- 
ive technologies. Adaptive testing shortens test times, allows children to get more 
questions that probe their understanding (rather than a lot of questions that are 
too easy or too hard), and requires the development of assessment scales that are 
more likely than traditional assessments to be aligned from grade to grade. The 
timeline for feedback to educators from computer adaptive testing is orders of mag- 
nitude shorter than the timeline for obtaining results from paper and pencil tests. 
Finally, the costs of computer adaptive testing when spread over a few years to am- 
ortize start-up investments in technology are lower than the costs of traditional test- 
ing. The federal government, in my view, should not stand in the way of the use 
of computer adaptive testing as it has done through the Department’s interpreta- 
tions of NCLB assessment requirements. And to the extent that discretionary funds 
are available, Congress should consider providing money to states to advance the 
use of this technology. 

As I indicated in my testimony, the RELs are not working well in that much of 
the work they produce is of little relevance to the needs of those responsible for 
schools in their regions. This has been the case for 40 years. My recommendation 
is that in lieu of authorizing RELs Congress should provide a voucher to state de- 
partments of education that could be used specifically to purchase data analytic 
services that use statewide longitudinal databases to address questions of imme- 
diate importance to decisions about education policy at the state level or among nu- 
merous school districts within the state. These analytic services could be obtained 
from any of a number of entities, including the existing RELs, that pass muster 
with the Institute of Education Sciences in terms of the quality of their research 
services. IES should retain a review function with respect to the analyses that are 
commissioned with the research vouchers to make sure than the conclusions 
reached are justified by the methods deployed. 

The national research and development centers serve an important function in 
providing for concentrated team-based research on education topics of national in- 
terest. However, it is a mistake, in my view, for Congress to dictate the topics on 
which the R&D centers should focus through authorization language per the current 
version of ESRA or the amounts that should be carved out for R&D centers vs. reg- 
ular competitive grants per appropriations language. The director and professional 
staff of the Institute of Education Sciences with the advice and consent of the Na- 
tional Board for Education Sciences is in the best position to know when there is 
both need and capacity in the field for an R&D center on a particular topic. In its 
efforts to comply with Congressional intent, IES frequently has held competitions 
for R&D centers on particular topics that generated only a few applications and 
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none of quality. This would not have happened if the hands of IES had not been 
tied on R&D centers through authorizing or appropriation language. 

Comprehensive centers are not part of ESRA and are not administered by IES, 
although frequently the contractor for a regional comprehensive center is the same 
as the contractor for the regional REL. The comprehensive centers are part of a 
patchwork of technical assistance providers that various offices of the Department 
contract with though a variety of program accounts. In my view the technical assist- 
ance entities that are funded through ED, including the comp centers, provide serv- 
ices of uncertain quality that are rarely driven by customer demand. Similar to my 
recommendation with regard to the RELs, I suggest that Congress consider shifting 
to a mechanism in which some portion of program funds that are appropriated pur- 
suant to ESEA, IDEA, Perkins, and other big budget programs is reserved for use 
by state departments of education to purchase technical assistance for implementa- 
tion of the federal education programs. The Department could be authorized to cre- 
ate a list of contractors who have demonstrated the capability of carrying out tech- 
nical assistance on particular topics. 

There are two important distinctions that are relevant to answering this question. 
The first is between evaluations of impact vs. implementation. Impact evaluations 
address the question of whether a program has a causal effect on the outcomes it 
is intended to influence. For example, an impact evaluation of Reading First would 
ask whether the reading achievement of participants in the program is accelerated 
compared to similar students who are not participants. An implementation evalua- 
tion, in contrast, would ask whether the funds for the program were expended as 
dictated by legislation and regulation. For example, were Reading First funds de- 
ployed to provide professional development for teachers as required in NCLB? 

The second distinction is between primary evaluations that are carried out by 
through the collection and analysis of original data, e.g., assessments of students 
carried out by the evaluation contractor vs. secondary evaluations that are based 
on summarizing and providing recommendations and conclusions based on a syn- 
thesis of results from previously published studies and other data sources. 

OMB and GAO do not carry out impact evaluations and rarely engage in primary 
evaluations. Rather they summarize what is known from primary data collections 
and from simple investigatory techniques such as engaging in interrogatories of pro- 
gram participants or program implementers. 

The Department’s Office of Planning, Evaluation, and Policy Development 
(OPEPD) has limited itself in recent years to implementation evaluations that are 
based on primary data collection and quick turn-around secondary evaluations that 
are of high relevance to the Secretary. OPEPD does not presently carry out impact 
evaluations, although it used to and nothing in statute prevents it from doing so. 

Private entities sometimes carry out impact evaluations of federal programs but 
these are typically conducted years after the program has been implemented and 
are based on available administrative data, e.g., existing school records, rather than 
primary data collection that is designed ahead of time to answer a range of planned 
questions. Thus the type of impact evaluation of a federal program that might be 
carried out, for example, by a university-based economist would only very rarely 
have the timeliness or the depth and breadth to answer questions that are impor- 
tant to Congress and the administration in decisions about program authorization 
and funding. 

Presently, only the Institute of Education Sciences carries out large scale impact 
evaluations of federal programs. None of the other entities listed in the question 
overlaps with IES in this function. This is a critical function that is being carried 
out well by IES. 

Presently, only IES and OPEPD carry out large scale primary implementation 
evaluations. OPEPD has generally carried out its implementation studies well, but 
that are significant inefficiencies in having two separate divisions of the Department 
involved in evaluating a single program. For example, program implementers may 
be required to answer a similar set of questions and respond to duplicative data re- 
quests from IES and OPEPD. Some of these problems of overlap cannot be solved 
by better coordination between IES and OPEPD because the activities are funded 
by different contracts that are awarded on different timetables to different contrac- 
tors. Further, there is always a legitimate concern about whether an office, OPEPD, 
that develops policy for and with the Secretary and that has no independence from 
the Secretary should be charged with evaluating whether programs the Secretary 
is charged with implementing are being carried out as intended in statute. For these 
reasons, it is my recommendation that IES be given the sole authority by Congress 
to carry out impact and implementation evaluations that are either required or per- 
mitted in program legislation. This has been the historical drift both within legisla- 
tion and in the division of responsibilities between IES and OPEPD as administra- 
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tively determined by the Department. It would be wise to cement this division of 
labor legislatively, in my view. In doing so Congress should designate funds specifi- 
cally for evaluation purposes rather than setting aside a percentage of funds in pro- 
gram authorizations to be used for “national activities, including evaluation.” The 
latter language is problematic in that it creates a competition between IES and the 
Department’s program offices for funds from the same pot, and it empowers the Sec- 
retary to throttle funds for evaluation activities that might expose performance 
issues with programs with which the administration is identified politically. In my 
view, all education programs with an annual price tag above a threshold of $20 mil- 
lion should be subject to an implementation and impact evaluation before they are 
reauthorized. These evaluations should be carried out by IES with funds specifically 
targeted to that purpose by Congress. 


[Whereupon, at 11:27 a.m., the subcommittee was adjourned.] 

o 



